We listed and answered frequently asked questions below. Please let us know which aspects are working well or could be improved via our 15-minute Survey Study. We appreciate your input!
A1: During labeling, videos need to play and loop automatically. However, if a mobile device has data saver enabled, videos will stop autoplay. Also, some mobile devices pause videos after users wake it up from sleeping mode. In order to enable autoplay, browsers require user interactions, which is why the system shows the dialog box.
A2: For each batch (16 videos) on the page, the system randomly placed several videos with known answers, also called gold standards. A batch will pass the quality check if you label these gold standards correctly.
A3: There can be two reasons. Firstly, videos that have closer times (e.g., 8 and 8:10 am) can look similar due to the same weather and lighting conditions. Secondly, gold standard videos for the quality check can appear again if you label many batches.
A4: We will use these labeled videos to train a deep neural network for detecting smoke emissions. While deep neural networks have been proven to outperform traditional models in various applications, training such large networks requires a considerable amount of labeled data, which is why we need volunteers' help.
A5: We selected and cropped several windows into videos from our camera network (as shown in the following image). Most videos are from the the Clairton Coke Works camera, and some videos are from the Edgar Thomson Steel Works camera. Each video contains 36 frames, which represent about 6 minutes in real-world time.
A6: This project is open-sourced on GitHub. Please feel free to reuse the code.
A8: When labeling smoke, this tool shows 16 videos at the same time. Older devices have difficulties in playing these videos, which results in poor user experiences.
A9: At least two volunteers will review each video. If their answers agree, the system marks the video according to the agreement. Otherwise, the video will be reviewed by an extra volunteer, and the result is aggregated based on majority voting.
A10: Smoke emissions in nighttime videos (captured by commercial digital cameras) are extremely tough for the computer to recognize due to insufficient light. We want to focus on training the computer to detect daytime smoke emissions first.