Updated: Dec 23, 20246 min read

Sentinel Sight 2.1 - Cloud and Local AI Vision in Home Assistant

Written by: Fixt
Sentinel Sight V2.1

Sentinel Sight 2


Shop Content
BLUEPRINT

Gain access to this content by buying it on our store

Prerequisites

OpenAI Logo
Ollama Logo

LLM Vision Integration

By valentinfrlch

Install

Open your Home Assistant instance and open a repository inside the Home Assistant Community Store.

Configure

Open your Home Assistant instance and start setting up a new integration.

Enable access to /config/www directory

homeassistant/configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /config/www

Install Visual Studio Code Add-on

Open your Home Assistant instance and show the dashboard of a Supervisor add-on.

Create Local To-Do

Open your Home Assistant and show de integration menu

Home Assistant Companion App

Home Assistant Internet Access

Only needed to be able to show the Image Preview and the Recording on the notifications using the Home Assistant Companion App

Open your Home Assistant instance and manage your systems network configuration.

Before you buy it

Open your Home Assistant instance and show your service developer tools.

Snapshot

Developer Tools/Services

service: camera.snapshot
data:
  filename: /config/www/tmp/snapshot_test1.jpg
target:
  entity_id: camera.YOUR_CAMERA

AI Provider

GPT4o

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: OpenAI
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg

Gemini

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: Google
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg

Ollama

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: Ollama
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg
  model: llava:latest

Notification

Mobile App Notification

Developer Tools/Services

service: notify.mobile_app_YOURDEVICE
data:
  title: Test Title
  message: Test AI message
  data:
    image: /local/tmp/snapshot_test1.jpg

Other Notify Service Notification

Record a Video

service: camera.record
data:
  filename: /config/www/tmp/video_snapshot_test.mp4
  duration: 15
target:
  entity_id: camera.YOUR_CAMERA

Send notification with Video

service: notify.YOUR_NOTIFY_SERVICE
data:
  title: Test Title
  message: Test Message
  data:
    video:
      file: /config/www/tmp/video_snapshot_test.mp4