Frame Sources

A frame source provides images to LectureSight for processing, typically from an overview camera.

The FrameSourceProvider service in the lecturesight-framesource-impl bundle provides the infrastructure responsible for managing FrameSource implementations. It discovers video input plugins and is responsible for setting up configured FrameSources with the proper input plugin.


Key Default Description
cv.lecturesight.framesource.i nput.mrl v4l:///d ev/video 0[width= 320;heig ht=240] MRL of the video input from the overview camera
cv.lecturesight.framesource.i nverted false If true, rotates the framesource 180 degrees. Used for cameras mounted inverted (upside-down)
cv.lecturesight.framesource.m axfps 60 Maximum fps rate at which frames should be read from the device.
cv.lecturesight.framesource.s napshot.file   Filename to which overview image snapshots should be saved periodically
cv.lecturesight.framesource.s napshot.interval 60 Interval in seconds to save overview image snapshots

The Media Resource Locator (MRL) has the following form:

type :// path [options]
Part Meaning
type the type of the input, determines which input plugin is used
path path to the input, usually a Linux device or file
options additional arguments for the input plugin

V4L Frame Source

The lecturesight-framesource-v4l bundle provides a FrameSource implementation for accquiring frames from Video4Linux and Video4Linux 2 devices such as webcams and frame grabbers.

Arguments for creation of a new FrameSource from this implementation can be provided in the FrameSource MRL. If an argument is not present in the MRL, the default value is taken from the configuration properties.


The type for this FrameSource implementation is v4l or v4l2. The path is the path to a Linux video device such as /dev/video

Available arguments are width, height, standard, channel, quality.


Video4Linux device /dev/video0 as input with QVGA resolution.


Video4Linux2 device /dev/video0 as input with QVGA resolution.



Key Default Description
cv.lecturesight.framesource.v 0 Default video input. Usually not used with USB webcams but rather with capture cards. This can be useful with capture cards, since they are by default set to tuner input and need to be set to composite (usually 1).
cv.lecturesight.framesource.v 4l.format YUYV YUYV or MJPEG for webcams
cv.lecturesight.framesource.v 4l.quality 0 Default encoding quality. Only used for devices that provide encoded video streams (such as MPEG2 or MJPEG). Value range depends on device driver.
cv.lecturesight.framesource.v 4l.resolution.width 320 Default width for input frames.
cv.lecturesight.framesource.v 4l.resolution.height 240 Default height for input frames.
cv.lecturesight.framesource.v 4l.standard 0 Default video standard. Usually not used with USB webcams but rather with capture cards. Which value indicates a certain standard (eg. PAL-X/NTSC) depends on the driver of the video device.

V4L2 Device Controls

It is also possible to set V4L2 device controls by including them in the option list. To find out which controls the v4l2 device supports, look at the device information logged by the V4LFrameGrabberFactory.createFrameGrabber method. For example this device has a control named Brightness which can be set in the range 0 to 255:

2017-12-12 06:31:43.587 INFO    CM Configuration Updater (Update: pid=org.apache.felix.fileinstall.235d4e28-9777-462b-93dd-d7a91af9d18e) V4LFrameGrabberFactory.createFrameGrabber() : Name: Brightness = 128 Type: CTRL_TYPE_SLIDER Values: [ 0 .. 255 ] increment: 1

To set this value in the MRL, add it to the option list:


GStreamer Frame Source

The lecturesight-framesource-gst bundle provides a Frame Source implementation that uses a user-defined GStreamer pipeline to capture frames. The pipeline definition is given in the format that is used in the gst-launch command. The implementation adds a color space element, a capsfilter and an appsink to the user-defined pipeline that are responsible for converting the frames to RGB format and hand them over to the system.


MRLs for this Frame Source contain the GStreamer pipeline definition in the format used in the gst-launch command.

gst://(gst-launch definition)[(options)]

Note: You must define a pipeline with at least two elements, otherwise the creation of the Frame Source will fail (issue LS-71).


drop – optional, default: true

Sets the drop property of the appsink that hands the frames over to the system.

For real-time frame sources such as cameras it is recommended to set this value to true so that the newest frames is uploaded to the GPU for video analysis.

When testing with a non-live frame source such as video files, drop may be set to false so that the system gets every frame for analysis.

More information on the appsink element and its drop property are available in the GStreamer documentation.


Consume an RTSP stream from an Axis IP camera:

cv.lecturesight.framesource.input.mrl=gst://rtspsrc location=rtsp:// ! rtph264depay ! avdec_h264 ! videoconvert

Use a V4L2 source:

cv.lecturesight.framesource.input.mrl=gst://v4l2src device=/dev/video0 ! ffmpegcolorspace [drop=true]

Use a gstreamer test pattern:

cv.lecturesight.framesource.input.mrl=gst://videotestsrc ! identity

RTPH264 Frame Source

The lecturesight-framesource-rtph264 bundle provides a FrameSource implementation that reads H264 video from an RTP stream. This is a special-purpose gstreamer pipeline designed to consume video from a RaspberryPi camera with minimal latency.



Videofile Frame Source

The lecturesight-framesource-videofile bundle provides a FrameSource implementation that reads frames from a video file using gstreamer.

It depends on the set of codecs installed in the host operating systems what formats are supported. Standard MPEG file formats should always be supported since they are included in most standard installations.

The type for this FrameSource implementation is file. The path is the path to a video file. There are no arguments for this FrameSource implementation.

Example, using the file /opt/ls/media/overview.mp4 as a frame source: