Computer vision takes the lion share of Machine learning world. Advanced AI use cases are dependent on video. Video-based ML projects are heavy in consuming resources with respect to processing power, disc space, streaming over net and internet consumption. We looked at Pafy that helps developers for converting video streaming to number of images. Machine Learning practitioners faces hurdles during collecting video data in the form of video files or using live streaming data. Hundreds of images are to be extracted from a video in order to identify person or object in the video at a particular time slot. This information is crucial to prepare the data and label them for training the model. Pafy Python module is of rescue and handy to the developers.
Computer vision projects are supposed to undergo massive pre-processing activities in order to transform raw blob data into useful applications. e.g.
- Image segmentation partitions an image into multiple regions or pieces to be examined separately.
- Object detection identifies a specific object in an image.
- Advanced object detection recognizes many objects in a single image: a football field, an offensive player, a defensive player, a ball and so on.
- Facial recognition is an advanced type of object detection that not only recognizes a human face in an image, but identifies a specific individual.
- Edge detection is a technique used to identify the outside edge of an object or landscape to better identify what is in the image.
- Pattern detection is a process of recognizing repeated shapes, colors and other visual indicators in images.
- Image classification groups images into different categories.
- Feature matching is a type of pattern detection that matches similarities in images to help classify them.
For example, consider a video with items in supermarket. In order to identify products in the video, video should be passed as an input to Pafy. It extracts images of products in the timeline of the video. Pafy objects relate to videos hosted online. They hold metadata such as title, view count, author and video ID. Stream objects relate to individual streams of an online video. They hold stream-specific data such as resolution, bitrate, and URL. Each Pafy object contains multiple stream objects.
As shown below, create a pafy object using the pafy.new() function, giving a streaming video URL as argument. Once an object is created, several data attributes are available like the author, category of video, video description text, number of likes, duration of the stream, keywords, published, rating, title, etc.
Using Pafy library you can hit streaming url and read blob stream and extract images that can be used in training. Below code demonstrates usage of pafy library and hands it over to cv2 to save frames to images.
This will help developers to collect and prepare data for training, testing model. Pafy facilitates streaming video to break up into series of images. A huge number of image repository for labelling which is a crucial part of machine learning pipeline. This reflects in quality of the model performance, accuracy and outcome.