Media Service Usage Instructions
Media Microservice 0.1.0
The Media Microservice processes RTSP video streams from the Camera Microservice, detecting people in the scene, and identifying their head orientation to infer gaze direction. It maintains consistent identity tracking across time and between cameras using multi-camera tracking and re-identification (Re-ID). It generates a Bird’s Eye View (BEV) visualization where:
Each detected person is represented by a dot.
The dot is green if the person is engaged (looking toward a target area).
The dot is red if the person is not engaged (looking away).
The engagement status (engaged/not engaged) for each person is retrieved from Redis (generated by the Engagement Analytics Microservice). On the other hand the Media Microservice publishes the position and head direction data of each person to Redis so that the Engagement Analytics service can perform its calculations.
The detection/direction messages use the DirectionSchemaGenerator from rrms-utils as follows:
{
"id": 1,
"cameraid": "camera",
"timestamp": "2025-01-01T00:00:00Z",
"width": 1920,
"height": 1080,
"detections": [
{
"objectid": "object1",
"position": {
"x": 10,
"y": 20
},
"direction": {
"x": 1,
"y": 0
}
}
]
}
The final BEV visualization, with overlaid person positions and engagement status, is streamed out over RTSP.
The service can optionally offer a brightness functionality to the BEV output via PVA. This functionality will be automatically added if the pvabrightness element is found on the system. If available, it can be controlled either via the /configuration or the dedicated /brightness request(check API)
Service Configuration
The service can be configured in 2 different ways: providing a configuration file via the –config_file CLI argument or via the /configuration request (see the service API documentation). In both cases, the configuration will be in JSON format as follows:
{
"inputs": [
{
"id": "camera0",
"index": 1
},
{
"id": "camera1",
"index": 0
}
],
"output": {
"port": 6062,
"resolution": {
"width": 1920,
"height": 1080
},
"mapping": "bev",
"bitrate": 4000000,
"brightness": 1.0
},
"cam_position": {
"x": 1166,
"y": 900
},
"head_pose_confidence": 0.1
}
Running the Service
The project is configured (via setup.py) to install the service with the name media. So to install it run:
pip install .
Then you will have the service with the following options:
The project is configured (via setup.py) to install the service with the name analytics. So to install it run:
```bash
pip install .
Then you will have the service with the following options:
media --help
usage: media [-h] [--config_file CONFIG_FILE] [--port PORT] [--host HOST] [--bev_calibration_file BEV_CALIBRATION_FILE]
options:
-h, --help show this help message and exit
--config_file CONFIG_FILE
JSON file with media initial configuration
--port PORT Port for server
--host HOST Server ip address
--bev_calibration_file BEV_CALIBRATION_FILE
BEV calibration file
To start the service in address 127.0.0.1 and port 5051 just run:
media --bev_calibration_file <path to the bev calibration file>
If you want to serve in a different port or address, use the –port and –host options. If you want to start with a configuration, you can use the –config-file argument to specify the path to your configuration.
gst-pva-brightness considerations
If PVA support is desired, before running the service, make sure you follow these steps to install the PVA application dependencies:
Install pvabrightness algorithm
cp libpvabrightness.so /usr/lib/aarch64-linux-gnu/
Update the PVA allow rules
cp pva_allowlist_default_gen2 /etc/pva/allow.d/my_allowlist
Register and install the available PVA SDK packages
cd <location of local_installers.deb>
sudo apt install ./pva-sdk-2.6-local-core_2.6.0_all.deb
sudo apt install ./pva-sdk-2.6-local-gen2_2.6.0_all.deb
sudo apt update
sudo apt install pva-sdk-2.6-l4t
Docker Support
Before starting with docker support make sure you have nvidia runtime in your system. Follow these instructions to have docker up and runing in your Jetson Board.
Build the docker image
We can build this microservice container using the Dockerfile in the docker directory. This includes a base NVIDA image and the dependencies to run the media microservice application.
First, we need to prepare the context directory for this build, please create a directory and include all the needed repositories (listed below). The Dockerfile will look for all the source code in the context directory and copy them to the container.
context/
├── gst-pva-brightness
├── gst-rr-track-id
├── gst-rtsp-sink
├── libpanorama
├── media
├── pva-sdk-and-algorithm-files
└── rrms-utils
NOTE: PVA support and gst-pva-brightness plugin will be installed if the –target pva flag is given during the image build. If given, it is also necessary to create a pva-sdk-and-algorithm-files directory in the context directory, and include in this directory the file listed below:
context/
└── pva-sdk-and-algorithm-files
├── libpvabrightness.so
├── pva_allowlist_default_gen2
├── pva-sdk-2.6-local-core_2.6.0_all.deb
└── pva-sdk-2.6-local-gen2_2.6.0_all.deb
Build the Docker image running the following command from the folder containing the Dockerfile and context directory:
Without PVA support
sudo docker build \
--network=host \
-f Dockerfile \
-t ridgerun/media-service:latest \
context/
With PVA support
sudo docker build \
--network=host \
--target pva \
-f Dockerfile \
-t ridgerun/media-service:latest \
context/
Change context to your context’s path and the tag (-t) to the name you want to give to your image.
To build the Docker image with RidgeRun’s products in evaluation mode, it will be necessary to clone the eval repository using git, for that we share the ssh-agent to Docker, use these commands to enable the ssh-agent:
eval $(ssh-agent)
ssh-add ~/.ssh/id_rsa
Verify that after the previous commands the $SSH_AUTH_SOCK variable is available:
echo $SSH_AUTH_SOCK
# Output
/tmp/ssh-XXXXDmihix/agent.2499
To enable evaluation mode, use the build argument ENABLE_EVAL, EVAL_MODE, EVAL_ITER, and EVAL_PERIOD. You can do this by executing the following command:
sudo docker build \
--build-arg ENABLE_EVAL=true \
--build-arg EVAL_MODE=iter \
--build-arg EVAL_ITER=9000 \
--build-arg EVAL_PERIOD=30 \
--ssh default=$SSH_AUTH_SOCK \
--network=host \
-f Dockerfile \
-t ridgerun/camera-service:latest \
camera-context/
EVAL_MODE options are iter and time. EVAL_ITER is the number of runs of the evaluation (used with the iter mode). EVAL_PERIOD is the number of days of the evaluation (used with the time mode). Notice we also added the argument –ssh default=$SSH_AUTH_SOCK
Launch the Container
The container can be launched by running the following command:
sudo docker run \
--runtime nvidia \
-it \
--network host \
--name media-service \
ridgerun/media-service:latest
You can modify the name you want to give to your container with the option –name.
Here we are creating a container called media-service that will start the media service application in the default address and port and using the default output resolution, and with no initial configuration. We need to start the pipeline by adding the configuration via request.
To start the service with an initial configuration file, or a different address or port has to be used, you can do it by running:
sudo docker run \
--runtime nvidia \
-it \
--network host \
-v <configuration file path on host>:/init_config.json \
--name media-service \
ridgerun/media-service:latest \
--config_file /init_config.json --host=<host to be used> --port=<port to be used>