1. Information

____________________________________________________________________________________

2. Prerequisites

Two Registrations:

____________________________________________________________________________________

3. Submission Format

Model Format: Participants can train their models using various libraries, such as PyTorch, ONNX, AI Model Efficiency Toolkit (AIMET) quantized models, and TensorFlow. Qualcomm AI Hub supports models trained with these libraries, and can directly compile them for mobile devices. Once the model is compiled using Qualcomm AI Hub, please follow the next two steps to submit it (compiled_job on AIHub) for evaluation and ranking.

There are two steps to complete the submission.

Step1: On Qualcomm AI Hub, share the access permission of the model that you want to submit with lowpowervision@gmail.com. It ensures that our evaluation server can access submitted models from Qualcomm AI Hub.

Step2: Fill up a submission form.

* Please refer to [Track 2 Sample Solution] for more details of submission. (TO BE UPDATED)

Share compile job

# IMPORTANT! You must share your compile job to LPCVC organizers thus we can pull and evaluate it.

compile_job.modify_sharing(add_emails=['lowpowervision@gmail.com'])

Please note: Your models will not be evaluated and ranked unless you complete both steps (step1 & step2). Each model requires a unique submission form because you must specify the Compile Job ID in the form.

____________________________________________________________________________________

4. Evaluation Details

4.1 Data

Training data: We do not limit specific training data for this competition. Participants are free to use any accessible datasets.

Test data: Hidden QEVD (TO BE QUPDATED)

Sample data: QEVD (TO BE QUPDATED)

4.2 Task

Goal: Classify the exercise action in a video clip

Model Input: An 8-frame video clip

Model Output: Classification logic

* Check the provided [Sample Solution (TO BE UPDATED)] for detailed input and output data format for the evaluation pipeline

4.3 Metrics

The evaluation is conducted in two stages:

4.4 Sample Solution

We accept [ResNet-2Plus1D] as a sample solution to better support potential participants. The corresponding latency (inference time) on the test data will be used as the reference to determine if the submitted solutions are valid or not.

4.5 Data Format in Evaluation: All test data will be used to evaluate the submitted solutions online using AIHub. Thus, we prepare the test data into a specific format to fit the requirements of the AIHub platform and QNN libraries. (TO BE UPDATED)

Input

Video

Data Format

Explanation

Sample data preparation code

Example data

____________________________________________________________________________________

5. Compile, Profile, Inference via AIHub

 Please refer to the provided [Sample Solution]for details of model compile, profile, and inference via AIHub.

Important! After the close of the submission window, the TOP-5 teams on the leaderboard will be contacted to confirm which model will be their final solution used for the evaluation on the whole test data. The converted ONNX model as well as detailed evaluation scripts should also be requested in addition to the QNN model shared via AIHub.

____________________________________________________________________________________

6. References