stable-2023-09-12

New models enabled (e2e tested via portable lib):

• MobileBert

Export API

• Two stage export API
  • We are in the process of moving away from exir.capture(): Please refer to this issue #290 for more details. Also look at the updated doc at https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/exporting_to_executorch.md
• exir.serialize
  • The exir.serialize module was renamed to exir._serialize and is now private
• transform()
  • For perform passes on the same dialect, use transform()

Runtime API

• Method
  • Added set_output_data_ptr(), which is a simpler and safer way to set the output buffers if they were not memory-planned
  • Program::load_method() now accepts an optional EventTracer parameter for non-global profiling and event data collection

Delegation API

• backend.init() and backend.execute() API changes.
  • BackendInitContext is a new added argument for backend.init and BackendExecutionContext is the new added argument for backend.execute().
  • How to rebase on these apis changes?
    • For backend.init, if runtime_allocator is not used, just mark context is not used with __ET_UNUSED. Otherwise, runtime_allocator can be accessed from the context.
    • For backend.execute, nothing has been added to context yet, just mark it with __ET_UNUSED directly. We’ll add event tracer for profiling via context soon.
• backend.preprocess() API changes
  • Updated backend.preprocess:
    • def preprocess( edge_program: ExportedProgram, compile_specs: List[CompileSpec], ) -> PreprocessResult
  • How to rebase on this API changes?
    • Wrap the result like PreprocessResult(processed_bytes=bytes)
• Partitioner.partition API changes
  • Updated Partition class definition. Move partition_tags from class attribute to be part of the ParititionResult.
    • def partition(self, graph_module: GraphModule) -> PartitionResult
  • How to rebase on this API change?
    • Wrap both partition_tags and the tagged_graph together as PartitionResult
• Example Quantizer and Delegate e2e demo
  • Added an example to show to add a quantizer and have it working with delegate to fully delegated a quantized MobileNetV2 model to the example backend.

XnnpackDelegate

• In an effort to align better with the rest of the Executorch AoT stack, XnnpackDelegate added preliminary support to also handle graphs exported with the canonical capture config (i.e. CaptureConfig.enable_aot=True and CaptureConfig._unlift=False)

SDK

• DelegateMappingBuilder to generate debug handle mapping AOT for delegates
  • https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/profiling_and_debugging_delegates.md
• BundledProgram enabled for usage with examples (more API changes to come in subsequent releases to improve usability, these will be breaking API changes)** **
  • Documentation:
    • https://github.com/pytorch/executorch/blob/stable/docs/website/docs/tutorials/bundled_program.md
  • Example code pointers:
    • https://github.com/pytorch/executorch/tree/stable/examples/bundled_executor_runner
    • https://github.com/pytorch/executorch/blob/stable/examples/export/export_bundled_program.py

Misc

• Linter enabled
• pytest enabled. Rerun pip install . to install pytest and other deps
• gtest enabled via buck, for example, run gtest for runtime/core
  • /tmp/buck2 test runtime/core/test/…
• Index operator rewrite:
  • Fixed bug related to null indices.
  • Implemented full Numpy’s advanced indexing functionality (now it is possible to use multidimensional indices, and masks that only index a subspace).
• Build/CMake
  • CMake release build mode with size optimization flags. We have an example in examples/selective_build/test_selective_build.sh