Updated: Aug 1, 20245 min read

Sentinel Sight 2.1 - Cloud and Local AI Vision in Home Assistant

Written by: Fixt
Sentinel Sight V2.1

Sentinel Sight 2


Shop Content
BLUEPRINT
Member Banner

Join our community to unlock exclusive
content!

  • As a Tinkerer 🤖, you’ll gain:
    • Member Content Access: You’ll have the hability to access Scripts, Automations, Templates and more directly from our website.
    • Unlock Latest Shop Content for FREE: Look for the MemberFree badge on the title.
      Once unlocked, it will remain free for you to access on our website (including future UPDATES!)
    • Ko-fi Shop Discount: Enjoy a 25% discount on any item in our Ko-fi shop.
    • Early Tutorial Access: Get early access to our content on our website.
    • Member Shoutout: Stick around until the end for a shoutout to our dedicated members
  • Your support is invaluable in allowing us to continue creating the tutorials you like 😁🏠🎉
  • Click on any of the buttons below to become a member, and then log in with the same email address.

Prerequisites

OpenAI Logo
Ollama Logo

LLM Vision Integration

By valentinfrlch

Install

Open your Home Assistant instance and open a repository inside the Home Assistant Community Store.

Configure

Open your Home Assistant instance and start setting up a new integration.

Enable access to /config/www directory

homeassistant/configuration.yaml

homeassistant:
  allowlist_external_dirs:
    - /config/www

Install Visual Studio Code Add-on

Open your Home Assistant instance and show the dashboard of a Supervisor add-on.

Create Local To-Do

Open your Home Assistant and show de integration menu

Home Assistant Companion App

Home Assistant Internet Access

Only needed to be able to show the Image Preview and the Recording on the notifications using the Home Assistant Companion App

Open your Home Assistant instance and manage your systems network configuration.

Before you buy it

Open your Home Assistant instance and show your service developer tools.

Snapshot

Developer Tools/Services

service: camera.snapshot
data:
  filename: /config/www/tmp/snapshot_test1.jpg
target:
  entity_id: camera.YOUR_CAMERA

AI Provider

GPT4o

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: OpenAI
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg

Gemini

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: Google
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg

Ollama

Developer Tools/Services

service: ha-llmvision.image_analyzer
data:
  provider: Ollama
  message: Tell me what you see in this image
  image_file: /config/www/tmp/snapshot_test1.jpg
  model: llava:latest

Notification

Mobile App Notification

Developer Tools/Services

service: notify.mobile_app_YOURDEVICE
data:
  title: Test Title
  message: Test AI message
  data:
    image: /local/tmp/snapshot_test1.jpg

Other Notify Service Notification

Record a Video

service: camera.record
data:
  filename: /config/www/tmp/video_snapshot_test.mp4
  duration: 15
target:
  entity_id: camera.YOUR_CAMERA

Send notification with Video

service: notify.YOUR_NOTIFY_SERVICE
data:
  title: Test Title
  message: Test Message
  data:
    video:
      file: /config/www/tmp/video_snapshot_test.mp4

I would love to hear your comments, or suggestions on this post.


Avatar of Fixt

Fixt is a Software Engineer passionate about making the world a better place through technology and automation.