Upload videos or set of images. Provide Youtube urls for automatic downloading. Browse & annotate uploaded videos. Ability to import pre-indexed datasets coming soon.
Perform scene detection, frame extraction on videos. Detect objects in frames/images, index entire frames/images, detected objects using deep learning algorithms.
Extracted objects, along with entire frames and crops, are indexed using deep features. The feature vectors are used for retrieval by query image.
Using Docker deploy on variety of machines with/without GPUs, local and cloud. Docker compose enables automated setup including database & RabbitMQ.
Metadata is stored in Postgres for easy extensibility. Files and vectors are stored in a consistent schema within a single folder.
Asynchronous processing via celery allows extraction, detection, indexing & query flows to be easily modified.
Visual Search using Nearest Neighbors algorithm as a primary interface
Upload videos, multiple images (zip file with folder names as labels)
Provide Youtube url to be automatically processed/downloaded by youtube-dl
Leverage pre-trained object recognition/detection, face recognition models for analysis and visual search.
Query against pre-indexed external datasets containing millions of images.
Metadata stored in Postgres, Operations performed asynchronously using celery tasks.
Separate queues and workers for selection of machines with different specifications (GPU vs RAM).
Videos, frames, indexes, numpy vectors stored in media directory, served through nginx
Explore data, manually run code & tasks without UI via a jupyter notebook explore.ipynb
Indexing using Google inception V3 trained on Imagenet
Alexnet using Pytorch (disabled by default; set ALEX_ENABLE=1 in environment variable to use)
YOLO 9000 (disabled by default; set YOLO_ENABLE=1 in environment variable to use)
Product images data (coming soon!)
Text detection models
Soundnet (requires extracting mp3 audio)
Deep Video analytics is implemented using Docker and works on Mac, Windows and Linux. Make sure you have latest version of Docker installed.
git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics cd DeepVideoAnalytics/docker && docker-compose up
You need to have latest version of Docker and nvidia-docker installed. The GPU Dockerfile is slightly different from the CPU version dockerfile.
pip install --upgrade nvidia-docker-compose git clone https://github.com/AKSHAYUBHAT/DeepVideoAnalytics cd DeepVideoAnalytics/docker_GPU && ./rebuild.sh nvidia-docker-compose up
We provide an AMI with all dependencies such as docker & nvidia drivers pre-installed. To use it start a P2.xlarge instance with AMI ID: ami-848f3d92 (N. Virginia) and ports 8000, 6006, 8888 open (preferably to only your IP). Run following commands after logging into the machine via SSH. After approximately 5 ~ 1 minutes the user interface will appear on port 8000 of the instance ip. AMI creation is documented here.
cd deepvideoanalytics && git pull cd docker_GPU && rm nvidia-docker-compose.yml && ./rebuild.sh && nvidia-docker-compose up
Security warning: The current GPU container uses nginx <-> uwsgi <-> django setup to ensure smooth playback of videos. However it runs nginix as root (within the container). Since you can modify AWS Security rules on-the-fly, allow inbound traffic only from your own IP address.
Following options can be specified in docker-compose.yml, or your environment to selectively enable/disable algorithms.
ALEX_ENABLE=1 (to use Alexnet with PyTorch. disabled by default)
YOLO_ENABLE=1 (to use YOLO 9000. disabled by default)
SCENEDETECT_DISABLE=1 (to disable scene detection. enabled by default)
RESCALE_DISABLE=1 (to disable rescaling of frame extracted from videos. enabled by default)