Issues with QNN tutorial

[kirklandsign]

Link to tuorial: https://pytorch.org/executorch/main/build-run-qualcomm-ai-engine-direct-backend.html

Repro: git checkout b26eee87949ac08e0e057e9c051ab72f58e95f6e Apr 9

• Get rid of -DBUCK in https://pytorch.org/executorch/main/build-run-qualcomm-ai-engine-direct-backend.html. No longer needed.
• Problems with CMake:

  executorch/examples/qualcomm/CMakeLists.txt

  Lines 105 to 106 in 856e085

  	${FLATCCRT_LIB}
  	gflags

  can't find libs.

ld.lld: error: unable to find library -letdump
ld.lld: error: unable to find library -lflatccrt_d

Need to add -DEXECUTORCH_BUILD_SDK=ON \:

cmake .. \
    -DBUCK2=buck2 \
    -DCMAKE_INSTALL_PREFIX=$PWD \
    -DEXECUTORCH_BUILD_SDK=ON \
    -DEXECUTORCH_BUILD_QNN=ON \
    -DQNN_SDK_ROOT=$QNN_SDK_ROOT \
    -DCMAKE_TOOLCHAIN_FILE=$ANDROID_NDK/build/cmake/android.toolchain.cmake \
    -DANDROID_ABI='arm64-v8a' \
    -DANDROID_NATIVE_API_LEVEL=23 \
    -B$PWD

• AOT part (Unrelated to QNN! I also see it with https://pytorch.org/executorch/main/getting-started-setup.html)
  See No such file or directory: executorch/exir/_serialize/program.fbs and scalar_type.fbs #2910 for full context. Overall, avoid using PYTHONPATH.

cd $EXECUTORCH_ROOT
python -m examples.qualcomm.scripts.deeplab_v3 -b build_android -m SM8550 --compile_only --download

FileNotFoundError: [Errno 2] No such file or directory: '/home/hsz/e3/executorch/exir/_serialize/program.fbs'

You can copy from schema/program.fbs + scalar_type.fbs to exir/_serialize/for now.

• Runtime: Now binary qnn_executor_runner depends onlibqnn_executorch_backend.so. So we need adb push build_android/lib/libqnn_executorch_backend.so ${DEVICE_DIR}
• Logs with actual run on 8450, using V69 libraries. SDK version is 2.19. Warnings OK?

I 00:00:00.001015 executorch:qnn_executor_runner.cpp:131] Model file /data/local/tmp/dlv3_qnn.pte is loaded.
I 00:00:00.001061 executorch:qnn_executor_runner.cpp:140] Using method forward
I 00:00:00.001073 executorch:qnn_executor_runner.cpp:188] Setting up planned buffer 0, size 9031680.
WARNING: linker: Warning: unable to normalize "$/data/local/tmp" (ignoring)
WARNING: linker: Warning: unable to normalize "$/data/local/tmp" (ignoring)
[INFO] [Qnn ExecuTorch]: create QNN Logger with log_level 2
[WARNING] [Qnn ExecuTorch]:  <W> Initializing HtpProvider

[WARNING] [Qnn ExecuTorch]:  <W> Function not called, PrepareLib isn't loaded!

[INFO] [Qnn ExecuTorch]: Initialize Qnn backend parameters for Qnn executorch backend type 2
[INFO] [Qnn ExecuTorch]: Caching: Caching is in RESTORE MODE.
[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


[WARNING] [Qnn ExecuTorch]:  <W> Function not called, PrepareLib isn't loaded!

[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


[WARNING] [Qnn ExecuTorch]:  <W> Function not called, PrepareLib isn't loaded!

[INFO] [Qnn ExecuTorch]: Running level=3 optimization.
I 00:00:00.179013 executorch:qnn_executor_runner.cpp:212] Method loaded.
E 00:00:00.232743 executorch:method.cpp:939] Overriding output data pointer allocated by memory plan is not allowed.
E 00:00:00.232777 executorch:qnn_executor_runner.cpp:261] ignoring error from set_output_data_ptr(): 0x2
E 00:00:00.267126 executorch:method.cpp:939] Overriding output data pointer allocated by memory plan is not allowed.
E 00:00:00.267143 executorch:qnn_executor_runner.cpp:261] ignoring error from set_output_data_ptr(): 0x2
I 00:00:00.267147 executorch:qnn_executor_runner.cpp:264] Inputs prepared.
[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


I 00:00:00.277254 executorch:qnn_executor_runner.cpp:413] Model executed successfully.
[INFO] [Qnn ExecuTorch]: Destroy Qnn backend parameters
[INFO] [Qnn ExecuTorch]: Destroy Qnn context
[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


[INFO] [Qnn ExecuTorch]: Destroy Qnn device
[WARNING] [Qnn ExecuTorch]:  <W> sg_stubPtr is not null, skip loadRemoteSymbols


[INFO] [Qnn ExecuTorch]: Destroy Qnn backend
[WARNING] [Qnn ExecuTorch]:  <W> qnnOpPackageManager: hexagon unload op package function pointer is nullptr!

[WARNING] [Qnn ExecuTorch]:  <W> Function not called, PrepareLib isn't loaded!

Also glitches with etdump:

F 00:00:00.189293 executorch:etdump_flatcc.cpp:170] In function check_ready_to_add_events(), assert failed ((etdump_gen_state == ETDumpGen_Adding_Allocators || etdump_gen_state == ETDumpGen_Block_Created)): ETDumpGen in an invalid state. Cannot add new events now.

Remove etdump part will make it run.

• Other feedback: Use cmake-out for linux, and cmake-android-out for android? It's already in gitignore.

[kirklandsign]

cc @cccclai