Note: Open weblinks on this page by Right clicking and selecting “Open link in a new tab”

Basic Information

•   Sponsor: Xilinx
•   Competition: Online submission through lpcv.ai during 2020/07/01-2020/07/31
•   Restriction: Each team can submit at most once every 24 hours
•   Vision Task: Image classification. ImageNet classification dataset - 1000 categories.
•   Hardware: Ultra96-V2 + Xilinx® Deep Learning Processor Unit (DPU)
•   Software: PYNQ + Vitis AI
•   Reimbursement policy: The first 20 teams that submit solutions will receive $270 per team.
•   Technical Support: Please join the online discussion forum lpcvc.slack.com.
•   Evaluation:

• Data: Training data are from ImageNet classification dataset available at the ILSVRC 2012 website. The evaluation data are holdout images collected following the ImageNet protocol.
• Evaluation Criteria: Submissions are evaluated based on the performance (latency, accuracy) of given task (image classification).
• Benchmark environment: All submissions will be evaluated using the Ultra96 board itself
• Evaluation criteria:
• Top-1 Accuracy baseline: should be > 68.5. Accuracy less than 68.5 will be disqualified.
• Winner will win based on latency and accuracy.
• (Accruacy-top1% - 68.5)/15)*0.4+(10/latencyms)*0.6

• Prizes: Winners will be announced online.

• 1st place: $1,500.
• 2nd place: $1,000.
• 3rd place: $500.

_____________________________________________________________________________________________________________________________

PYNQ-DPU demo:

_____________________________________________________________________________________________________________________________

DPU_PYNQ.mp4

_____________________________________________________________________________________________________________________________

Update: DPU on PYNQ v2.5 with Vitis AI 1.1 support is now available!

Read the Announcement Here

Now participants can use this Vitis AI 1.1 based flow in lieu of the Vitis AI 1.0 build flow below

DPU on PYNQ v2.5 Ultra96v2 Image (Build Scripts Download link)

_____________________________________________________________________________________________________________________________

The following steps help users make their own DPU design work with Ultra96v2 PYNQ image v2.5. To do that, we need to first upgrade PYNQ v2.5 image to be compatible with Vitis AI 1.0. Then we will introduce the steps to rebuild the hardware design. Finally we will cover steps to run DPU applications on the Ultra96 v2 board.

_____________________________________________________________________________________________________________________________

Prerequisite

_____________________________________________________________________________________________________________________________

We have the following assumptions:

1. As a starting point, we assume you have flashed your SD card with Ultra96v2 PYNQ image v2.5.
2. You can boot up the board without any issue. It is recommended for you to use bigger power bricks; for example, 4A power brick.
3. You have an Internet connection from the board. If you have a USB-Ethernet dongle, connect that to the USB port of Ultra96, that will give    you a stable wired connection.
4. Your host machine (Linux) should have Vitis 2019.2 and XRT 2019.2 installed.

To make sure the image is correct (image v2.5), run the following on the board:

This should show:

_____________________________________________________________________________________________________________________________

Step 1: Upgrade PYNQ Image on Ultra96v2

_____________________________________________________________________________________________________________________________

The first step is to upgrade your image so we can install some libraries required by Vitis AI.

On your Ultra96 board, run su to use superuser. Password is xilinx. Copy the ultra96 folder over to your board.

Go to the ultra96 folder, and run:

The upgrade process can take up to 4 hours so please be patient.

At the same time, you can proceed to the next few steps.

By default, the upgrade.sh script will install / update the following packages:

• pynq: Install pynq 2.5.1 from source distribution.
• xrt: Xilinx Run Time (XRT) will be upgraded to tag 2019.2_RC2.
• ubuntu_pkg: Install devmem2 and at-spi2-core from Ubuntu repository; remove the old opencv package.
• opencv: Build and install opencv 3.4.3 from source, which takes around 60 mins.
• protobuf: Build and install protobuf 3.6.1 from source, which takes around 60 mins.
• jsonc: Build and install json-c 0.13.1 from source, which takes around 10 mins.
• dpu_clk: Install a small Python script to control DPU clocks.
• dnndk: Download and install DNNDK run time library for DPU.

Users can also disable any of them by setting the option to 0.

For example:

_____________________________________________________________________________________________________________________________

Step 2: Build Hardware Design on Host

Note: You can skip this step and download pre-compiled files here

_____________________________________________________________________________________________________________________________

While your board is being upgraded, we can take a look at the hardware design.

On your host machine, copy the host folder over to your host.

Source proper Vitis 2019.2 settings. Go to the host folder, and run

The above step can take up to 2 hours. Please be patient.

The build.sh does the following things:

• Build a bare minimum Vitis platform dpu.
• Build the DPU design based on Vitis AI TRD and the Vitis platform dpu.
• Copy overlay files (dpu.hwh, dpu.bit, and dpu.xclbin) out.

The released Vitis AI TRD is good for ZCU102 and ZCU104. You can check what we have changed in build.sh for Ultra96 so that the DPU IP can be successfully implemented on our Vitis platform. In general, Ultra96 has a smaller PL, so we can only target smaller DPU's on this platform.

After the building process is done, you should see some messages indicating the building is successful. 3 overlay files will be copied to the folder dpu_overlay: dpu.hwh, dpu.bit, and dpu.xclbin.

Now you have the DPU overlay files which can work with PYNQ!

Keep those 3 files in good hands.

Note: As long as we have those 3 files, we are done; please do NOT replace PYNQ SD card content!

_____________________________________________________________________________________________________________________________

Step 3: Build DPU Models on Host

Note: You can download pre-compiled example model binaries here

_____________________________________________________________________________________________________________________________

On your host machine, go to your host folder. If you have already run the build.sh in the previous step, you should be able to see (1) a Vitis AI Github repository has been cloned here, and (2) the dpu.hwh file has been generated under dpu_overlay folder.

If you have not installed docker on your host machine, please refer to the Vitis AI getting started page to install docker. If you have docker already installed, we can run the following commands now.

The docker_run.sh will download a Vitis AI docker image.

It may take a long time since the image is about 8GB. After the download is complete, the docker_run.sh script will help you log onto that docker image. The Vitis AI docker image gives users access to the Vitis AI utilities and compilation tools. If you have run docker_run.sh before, it will simply launch the docker image without downloading again. Once you are in the docker environment, you can run the compile.sh script that we provided.

Users can also check this compile.sh and make some adjustments based on their needs. In summary, this script does the following things:

• Adding Ultra96 support: we prepared the Ultra96.dcf and Ultra96.json files using the dpu.hwh file.
• Downloading a model from model zoo: as an example, we download the resnet50 model for image classification.
• Compiling the model: we compile the model into a *.elf file.

If everything is successful, you should see a screen as shown below.

A new model file dpu_resnet50_0.elf should appear in your local working directory; you can deploy this file on the Ultra96 board later.

After you are done with the docker environment, type in:

to exit.

For more information such as training your own model, please refer to the Vitis AI user guide and Vitis AI Tutorials.

_____________________________________________________________________________________________________________________________

Step 4: Run Application on Ultra96v2

_____________________________________________________________________________________________________________________________

On your Ultra96 board, go to your jupyter notebook root folder, and run

Put the following files in this pynq-dpu folder:

1. The overlay files (dpu.bit, dpu.hwh, dpu.xclbin)
2. The Python file app/dpu.py.
3. Any DPU model that you have compiled in Step 3 (e.g. dpu_resnet50_0.elf).

After that, you can create a new Jupyter notebook file in the same folder; add and run the following code:

And check the DPU information in a Jupyter notebook cell.

If you have compiled the dpu_resnet50_0.elf as introduced in Step 3, you can load it now:

Then you should be able to play with a couple of applications!

For example, you can try to rebuild and run the resnet50 application on Vitis AI Github. The basic steps are:

1. Adjust the KRENEL_CONV in the source file to resnet50_0; this is consistent with what is shown as DPUKernel in the screenshot above.
2. In the terminal, run make to rebuild the resnet50 application.
3. In the terminal, run the executable to classify a specific image (e.g., ./resnet50 img/bellpeppe-994958.JPEG).

There are other applications and models that you can explore in the Vitis AI Github.

_____________________________________________________________________________________________________________________________

References _____________________________________________________________________________________________________________________________

We list some of the useful references:

• Vitis AI Github
• Vitis AI User Guide
• Vitis AI TRD
• Vitis AI Model Zoo
• Vitis Embedded Platform Source
• PYNQ webpage
• DPU

_____________________________________________________________________________________________________________________________

Submission Format: _____________________________________________________________________________________________________________________________

Please download the submission notebook and scripts here. The steps to run the submission scripts are in the README file in the download.