feat(ecosystem): compliant Praxis/Hexis integration + vendored build
Bring the Nexus/Praxis/Hexis integration in line with MAVEN_ECOSYSTEM_ARCHITECTURE.md: - Praxis over HTTP: drop the in-process praxis.db open (praxisstore/ praxistools) and call praxisd's /api/v1/tools/* API via a new praxisClient. Honors the "no component reads another's DB" invariant (AC#12). PraxisConfig.DBPath -> URL. - Hexis confirmation gate: mutating capabilities (ReadOnly=false) now park a bound pendingHexis confirmation and require a spoken "да" before executing; read-only run immediately (AC#7, no auto attention->action). - Capability safety: >1 verb match is ambiguous -> ask instead of firing the first; ambiguous Nexus resolution asks for clarification (AC#2). - Correlation IDs on Hexis execute, recorded in the cross-service trace. - Bug: importance arrives as JSON float64 over HTTP, not int. - Tests: confirm-gate, decline, read-only, and ambiguity paths. Build: vendor/ bakes in the hexis client (replace-directed at a sibling repo outside the Docker context); Dockerfile builds from vendor and no longer `go mod download`s the unreachable replace paths. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
kami
@@ -0,0 +1,3 @@
|
||||
onnxruntime_c_api.h linguist-vendored
|
||||
onnxruntime_ep_c_api.h linguist-vendored
|
||||
|
||||
+136
@@ -0,0 +1,136 @@
|
||||
Contribution Guidelines
|
||||
=======================
|
||||
|
||||
This library began as a personal project, and is primarily still maintained as
|
||||
such. The following list of guidelines is not necessarily exhaustive, and,
|
||||
ultimately, any contribution is subject to the maintainer's discretion. That
|
||||
being said, contributions are welcome, and most recent new features have been
|
||||
added by users who need them!
|
||||
|
||||
Coding Style
|
||||
------------
|
||||
|
||||
- Go code must be formatted using the official `gofmt` tool.
|
||||
|
||||
- C code should adhere to the portions of Google's C++ style guide that
|
||||
apply to C.
|
||||
|
||||
- If at all possible, any Go or C code should have at most 80 character lines.
|
||||
(This may not be enforced very strictly.)
|
||||
|
||||
- Purely stylistic changes are unlikely to be accepted. Instead, the
|
||||
maintainer or other contributers may make small stylistic adjustments to
|
||||
surrounding code as part of other contributions.
|
||||
|
||||
- Attempt to mimic the existing style of the surrounding code.
|
||||
|
||||
|
||||
Documentation
|
||||
-------------
|
||||
|
||||
- All Go types, public-facing functions, and nontrivial internal functions
|
||||
must include a comment on their intended usage, to be parsed by godoc.
|
||||
|
||||
- As per the google C++ style guide, all C functions must be documented with a
|
||||
comment as well. If a C function is defined in a header file, the comment
|
||||
should appear with the definition in the header. If it's a static function
|
||||
in a `.c` file, the comment should appear with the function definition.
|
||||
|
||||
|
||||
Tests
|
||||
-----
|
||||
|
||||
- All new features and bugfixes must include a basic unit test (in
|
||||
`onnxruntime_test.go`) to serve as a sanity check.
|
||||
|
||||
- If a test is for a platform-dependent or execution-provider-dependent
|
||||
feature, the test must be skipped if run on an unsupported system.
|
||||
|
||||
- No tests should panic. Always check errors and fail rather than allowing
|
||||
tests to panic.
|
||||
|
||||
- Every change must ensure that `go test -v -bench=.` passes.
|
||||
|
||||
- Every test failure should be accompanied by a message containing the reason,
|
||||
either using `t.Logf()`, `t.Errorf()`, or `t.Fatalf()`.
|
||||
|
||||
|
||||
Adding New Files
|
||||
----------------
|
||||
|
||||
- Apart from testing data, try not to add new source files.
|
||||
|
||||
- Do not add third-party code or headers. The only exceptions for now are
|
||||
`onnxruntime_c_api.h` and `onnxruntime_ep_c_api.h`.
|
||||
|
||||
- No C++ at all. Developing Go-to-C wrappers is annoying enough as it is.
|
||||
|
||||
- Do not add any new `onnxruntime` shared libraries under `test_data`. I know
|
||||
there are additional platforms that would be nice to include (such as
|
||||
`x86_64` Linux), but I do not want this project turning into an unofficial
|
||||
distribution channel for onnxruntime libraries. It also clogs up the git
|
||||
repo with large files, and increases the size of the history every time
|
||||
these files are updated. The libraries that are included were only intended
|
||||
to allow a majority of users to run `go test -v -bench=.` without further
|
||||
setup or modification. Currently: amd64 Windows, arm64 Linux (I wish I
|
||||
hadn't included this!), and arm64 osx. All other users must set the
|
||||
`ONNXRUNTIME_SHARED_LIBRARY_PATH` environment variable to a valid path
|
||||
to the correct `onnxruntime` shared library file prior to running tests.
|
||||
|
||||
- If you need to add a .onnx file for a test, place both the .onnx file
|
||||
_and_ the script used to generate it into `test_data/`.
|
||||
|
||||
- Keep any testing .onnx files as small as possible.
|
||||
|
||||
- Without a good reason (i.e., implementing an entire class of APIs such as
|
||||
training), avoid adding new Go files---just add to `onnxruntime_go.go`.
|
||||
|
||||
|
||||
Dependencies
|
||||
------------
|
||||
|
||||
- Avoid Go or C dependencies outside of the language's standard libraries.
|
||||
This package currently does not depend on any third-party Go modules, and
|
||||
it would be great to keep it this way.
|
||||
|
||||
- Python scripts within `test_data/` can use whatever dependencies they need,
|
||||
because end users should not be required to run the python files, and the
|
||||
`.onnx` file they produce should already be included.
|
||||
|
||||
|
||||
C-Specific Stuff
|
||||
----------------
|
||||
|
||||
- Minimize Go management of C-allocated memory as much as possible. For
|
||||
example, see the `convertORTString` function on `onnxruntime_go.go`, which
|
||||
copies a C-allocated string into a garbage-collected go `string`.
|
||||
|
||||
- If you need to use a `OrtAllocator` in onnxruntime's C API, always use the
|
||||
default `OrtAllocator` returned by
|
||||
`ort_api->GetAllocatorWithDefaultOptions()`.
|
||||
|
||||
- ONNXRuntime APIs requiring file paths typically use `ORTCHAR_T*`
|
||||
strings. On Linux/OSX/etc, these should be UTF-8, but on Windows they will
|
||||
be wide-character strings. (Our tricks with `#include` to make them look
|
||||
like `char*` to C code even on Windows, but the DLL still expects a
|
||||
`wchar_t*`.) The important takeaway: when passing `ORTCHAR_T*`
|
||||
values to the onnxruntime C API, use the `createOrtCharString(...)`
|
||||
function. It converts a Go string to a C string, but unlike `C.CString`, it
|
||||
will do UTF8 to UTF16 conversion on Windows. (On Linux, it simply wraps
|
||||
`C.CString`.)
|
||||
|
||||
|
||||
A Few Notes on Organization
|
||||
---------------------------
|
||||
|
||||
- The `onnxruntime` C API uses a struct containing function pointers. Cgo
|
||||
can't directly invoke functions via pointers, so `onnxruntime_wrapper.c`
|
||||
(along with the associated header file) are used to provide top-level C
|
||||
functions that call the function pointers within the `OrtApi` struct.
|
||||
|
||||
- Linux and OSX use `dlopen` to load the onnxruntime shared library, but this
|
||||
isn't possible on Windows, which instead can use the `syscall.LoadLibrary()`
|
||||
function from Go's standard library. This different behavior is locked
|
||||
behind build constraints in `setup_env.go` and `setup_env_windows.go`,
|
||||
respectively.
|
||||
|
||||
+19
@@ -0,0 +1,19 @@
|
||||
Copyright (c) 2023 Nathan Otterness
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
+219
@@ -0,0 +1,219 @@
|
||||
Cross-Platform `onnxruntime` Wrapper for Go
|
||||
===========================================
|
||||
|
||||
About
|
||||
-----
|
||||
|
||||
This library seeks to provide an interface for loading and executing neural
|
||||
networks from Go(lang) code, while remaining as simple to use as possible.
|
||||
|
||||
A few example applications using this library can be found in the
|
||||
[`onnxruntime_go_examples` repository](https://github.com/yalue/onnxruntime_go_examples).
|
||||
|
||||
The [onnxruntime](https://github.com/microsoft/onnxruntime) library provides a
|
||||
way to load and execute ONNX-format neural networks, though the library
|
||||
primarily supports C and C++ APIs. Several efforts exist to have written
|
||||
Go(lang) wrappers for the `onnxruntime` library, but as far as I can tell, none
|
||||
of these existing Go wrappers support Windows. This is due to the fact that
|
||||
Microsoft's `onnxruntime` library assumes the user will be using the MSVC
|
||||
compiler on Windows systems, while CGo on Windows requires using Mingw.
|
||||
|
||||
This wrapper works around the issues by manually loading the `onnxruntime`
|
||||
shared library, removing any dependency on the `onnxruntime` source code beyond
|
||||
the header files. Naturally, this approach works equally well on non-Windows
|
||||
systems.
|
||||
|
||||
Additionally, this library uses Go's recent addition of generics to support
|
||||
multiple Tensor data types; see the `NewTensor` or `NewEmptyTensor` functions.
|
||||
|
||||
Several accelerated execution providers (including TensorRT, CUDA and CoreML)
|
||||
are tested and confirmed to work with `onnxruntime_go`. The "Requirements"
|
||||
portion of this README (below) has a few more details.
|
||||
|
||||
|
||||
Note on onnxruntime Library Versions
|
||||
------------------------------------
|
||||
|
||||
At the time of writing, this library uses version 1.26.0 of the onnxruntime
|
||||
C API headers. So, it will probably only work with version 1.26.0 of the
|
||||
onnxruntime shared libraries, as well. If you need to use a different version,
|
||||
or if I get behind on updating this repository, updating or changing the
|
||||
onnxruntime version should be fairly easy:
|
||||
|
||||
1. Replace the `onnxruntime_c_api.h` and `onnxruntime_ep_c_api.h` files with
|
||||
the versions corresponding to the onnxruntime version you wish to use.
|
||||
|
||||
2. Replace the `test_data/onnxruntime.dll` (or `test_data/onnxruntime*.so`,
|
||||
`test_data/onnxruntime*.dylib`) file with the version corresponding to the
|
||||
onnxruntime version you wish to use.
|
||||
|
||||
3. (If you care about DirectML support) Verify that the entries in the
|
||||
`DummyOrtDMLAPI` struct in `onnxruntime_wrapper.c` match the order in which
|
||||
they appear in the `OrtDmlApi` struct from the `dml_provider_factory.h`
|
||||
header in the official repo. See the comment on this struct in
|
||||
`onnxruntime_wrapper.c` for more information.
|
||||
|
||||
Note that both the C API headers and the shared library files are available to
|
||||
download from the releases page in the
|
||||
[official repo](https://github.com/microsoft/onnxruntime). Download the archive
|
||||
for the release you want to use, and extract it. The header files are located
|
||||
in the "include" subdirectory, and the shared library will be located in the
|
||||
"lib" subdirectory. (On Linux systems, you'll need the version of the .so with
|
||||
the appended version numbers, e.g., `libonnxruntime.so.1.26.0`, and _not_ the
|
||||
`libonnxruntime.so`, which is just a symbolic link.) The archive will contain
|
||||
several other files containing C++ headers, debug symbols, and so on, but you
|
||||
shouldn't need anything other than the single onnxruntime shared library and
|
||||
the two `_c_api.h` header files. (The exception is if you're wanting to enable
|
||||
GPU support, where you may need other shared-library files, such as
|
||||
`execution_providers_cuda.dll` and `execution_providers_shared.dll` (or their
|
||||
equivalents for Linux or OSX).
|
||||
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
To use this library, you'll need a version of Go with cgo support. You'll also
|
||||
need a copy of the correct version of the onnxruntime shared library or DLL for
|
||||
your operating system and architecture. Prior to initializing
|
||||
`onnxruntime_go`, you need to provide a path to this shared library. See the
|
||||
first couple lines (i.e., `ort.SetSharedLibraryPath(...)`) of the following
|
||||
example.
|
||||
|
||||
If you want to use CUDA, you'll need to be using a version of the onnxruntime
|
||||
shared library with CUDA support, as well as be using a CUDA version supported
|
||||
by the underlying version of your onnxruntime library. For example, version
|
||||
1.23.2 of the onnxruntime library only supports CUDA versions 12.x. See
|
||||
[the onnxruntime CUDA support documentation](https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html)
|
||||
for more specifics.
|
||||
|
||||
Similarly to CUDA, other execution providers have their own separate
|
||||
requirements. All of these requirements are too numerous to document in this
|
||||
README. Please ensure that you are successfully able to use your execution
|
||||
provider of choice in a python script before raising issues about it here.
|
||||
|
||||
|
||||
Example Usage
|
||||
-------------
|
||||
|
||||
The full documentation can be found at [pkg.go.dev](https://pkg.go.dev/github.com/yalue/onnxruntime_go).
|
||||
|
||||
Additionally, several example command-line applications complete with necessary
|
||||
networks and data can be found in the
|
||||
[`onnxruntime_go_examples` repository](https://github.com/yalue/onnxruntime_go_examples).
|
||||
|
||||
The following example illustrates how this library can be used to load and run
|
||||
an ONNX network taking a single input tensor and producing a single output
|
||||
tensor, both of which contain 32-bit floating point values. Note that error
|
||||
handling is omitted; each of the functions returns an err value, which will be
|
||||
non-nil in the case of failure.
|
||||
|
||||
```go
|
||||
import (
|
||||
"fmt"
|
||||
ort "github.com/yalue/onnxruntime_go"
|
||||
"os"
|
||||
)
|
||||
|
||||
func main() {
|
||||
// This line _may_ be optional; by default the library will try to load
|
||||
// "onnxruntime.dll" on Windows, and "onnxruntime.so" on any other system.
|
||||
// For stability, programs should always set this explicitly.
|
||||
ort.SetSharedLibraryPath("path/to/onnxruntime.so")
|
||||
|
||||
err := ort.InitializeEnvironment()
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
defer ort.DestroyEnvironment()
|
||||
|
||||
// For a slight performance boost and convenience when re-using existing
|
||||
// tensors, this library expects the user to create all input and output
|
||||
// tensors prior to creating the session. If this isn't ideal for your use
|
||||
// case, see the DynamicAdvancedSession type in the documnentation, which
|
||||
// allows input and output tensors to be specified when calling Run()
|
||||
// rather than when initializing a session.
|
||||
inputData := []float32{0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9}
|
||||
inputShape := ort.NewShape(2, 5)
|
||||
inputTensor, err := ort.NewTensor(inputShape, inputData)
|
||||
defer inputTensor.Destroy()
|
||||
// This hypothetical network maps a 2x5 input -> 2x3x4 output.
|
||||
outputShape := ort.NewShape(2, 3, 4)
|
||||
outputTensor, err := ort.NewEmptyTensor[float32](outputShape)
|
||||
defer outputTensor.Destroy()
|
||||
|
||||
session, err := ort.NewAdvancedSession("path/to/network.onnx",
|
||||
[]string{"Input 1 Name"}, []string{"Output 1 Name"},
|
||||
[]ort.Value{inputTensor}, []ort.Value{outputTensor}, nil)
|
||||
defer session.Destroy()
|
||||
|
||||
// Calling Run() will run the network, reading the current contents of the
|
||||
// input tensors and modifying the contents of the output tensors.
|
||||
err = session.Run()
|
||||
|
||||
// Get a slice view of the output tensor's data.
|
||||
outputData := outputTensor.GetData()
|
||||
|
||||
// If you want to run the network on a different input, all you need to do
|
||||
// is modify the input tensor data (available via inputTensor.GetData())
|
||||
// and call Run() again.
|
||||
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
Deprecated APIs
|
||||
---------------
|
||||
|
||||
**Typed `Session[t]`:** Older versions of this library used a typed
|
||||
`Session[T]` struct to keep track of sessions. In retrospect, associating type
|
||||
parameters with Sessions was unnecessary, and the `AdvancedSession` type, along
|
||||
with its associated APIs, was added to rectify this mistake. For backwards
|
||||
compatibility, the old typed `Session[T]` and `DynamicSession[T]` types are
|
||||
still included and unlikely to be removed. However, they now delegate their
|
||||
functionality to `AdvancedSession` internally. New code should always favor
|
||||
using `AdvancedSession` directly.
|
||||
|
||||
**Onnxruntime's training API:** The training API has been deprecated as of
|
||||
onnxruntime version 1.20. Rather than continuing to maintain wrappers for a
|
||||
deprecated API, `onnxruntime_go` has replaced the wrapper functions for the
|
||||
training API with stubs that return an error. Users who need to continue to
|
||||
use the training API will need to use an older version. For example the
|
||||
following versions should be compatible with training:
|
||||
|
||||
- Version `v1.12.1` of `onnxruntime_go`, and
|
||||
- Version 1.19.2 of `onnxruntime`.
|
||||
|
||||
|
||||
Running Tests and System Compatibility for Testing
|
||||
--------------------------------------------------
|
||||
|
||||
Navigate to this directory and run `go test -v`, or optionally
|
||||
`go test -v -bench=.`. All tests should pass; tests relating to CUDA or other
|
||||
accelerator support will be skipped on systems or onnxruntime builds that don't
|
||||
support them.
|
||||
|
||||
Currently, this repository includes a copy of the onnxruntime shared libraries
|
||||
for a few systems, including AMD64 windows, ARM64 Linux, and ARM64 darwin.
|
||||
These should allow tests to pass on those systems without users needing to copy
|
||||
additional libraries beyond cloning this repository. In the future, however,
|
||||
this may change if support for more systems are added or removed.
|
||||
|
||||
You may want to use a different version of the `onnxruntime` shared library for
|
||||
a couple reasons. In particular:
|
||||
|
||||
1. The included shared library copies do not include support for CUDA or other
|
||||
accelerated execution providers, so CUDA-related tests will always be
|
||||
skipped if you use the default libraries in this repo.
|
||||
|
||||
2. Many systems, including AMD64 and i386 Linux, and x86 osx, do not currently
|
||||
have shared libraries included in `test_data/` in the first place. (I would
|
||||
like to keep this directory, and the overall repo, smaller by keeping the
|
||||
number of shared libraries small.)
|
||||
|
||||
If these or other reasons apply to you, the test code will check the
|
||||
`ONNXRUNTIME_SHARED_LIBRARY_PATH` environment variable before attempting to
|
||||
load a library from `test_data/`. So, if you are using one of these systems or
|
||||
want accelerator-related tests to run, you should set the environment variable
|
||||
to the path to the onnxruntime shared library. Afterwards, `go test -v` should
|
||||
run and pass.
|
||||
+225
@@ -0,0 +1,225 @@
|
||||
package onnxruntime_go
|
||||
|
||||
// This file contains code and types that we maintain for compatibility
|
||||
// purposes, but is not expected to be regularly maintained or udpated.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
)
|
||||
|
||||
// #include "onnxruntime_wrapper.h"
|
||||
import "C"
|
||||
|
||||
// DEPRECATED: This type was written with a type parameter despite the fact
|
||||
// that a type parameter is not necessary for any of its underlying
|
||||
// implementation. It is preserved only for compatibility with older code, and
|
||||
// new users should use AdvancedSession instead. Despite the name,
|
||||
// AdvancedSession is equally simple to use and far more flexible.
|
||||
type Session[T TensorData] struct {
|
||||
// We now delegate all of the implementation to an AdvancedSession here.
|
||||
s *AdvancedSession
|
||||
}
|
||||
|
||||
// DEPRECATED: See the notes on Session[T]. Use DynamicAdvancedSession instead.
|
||||
type DynamicSession[In TensorData, Out TensorData] struct {
|
||||
s *DynamicAdvancedSession
|
||||
}
|
||||
|
||||
// DEPRECATED: See the notes on Session[T]. Use NewAdvancedSessionWithONNXData
|
||||
// instead.
|
||||
func NewSessionWithONNXData[T TensorData](onnxData []byte, inputNames,
|
||||
outputNames []string, inputs, outputs []*Tensor[T]) (*Session[T], error) {
|
||||
// Unfortunately, a slice of pointers that satisfy an interface don't count
|
||||
// as a slice of interfaces (at least, as I write this), so we'll make the
|
||||
// conversion here.
|
||||
tmpInputs := make([]Value, len(inputs))
|
||||
tmpOutputs := make([]Value, len(outputs))
|
||||
for i, t := range inputs {
|
||||
tmpInputs[i] = t
|
||||
}
|
||||
for i, t := range outputs {
|
||||
tmpOutputs[i] = t
|
||||
}
|
||||
s, e := NewAdvancedSessionWithONNXData(onnxData, inputNames, outputNames,
|
||||
tmpInputs, tmpOutputs, nil)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
return &Session[T]{
|
||||
s: s,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// DEPRECATED: See the notes on Session[T]. Use
|
||||
// NewDynamicAdvancedSessionWithONNXData instead.
|
||||
func NewDynamicSessionWithONNXData[in TensorData, out TensorData](onnxData []byte,
|
||||
inputNames, outputNames []string) (*DynamicSession[in, out], error) {
|
||||
s, e := NewDynamicAdvancedSessionWithONNXData(onnxData, inputNames,
|
||||
outputNames, nil)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
return &DynamicSession[in, out]{
|
||||
s: s,
|
||||
}, nil
|
||||
}
|
||||
|
||||
// DEPRECATED: See the notes on Session[T]. Use NewAdvancedSession instead.
|
||||
func NewSession[T TensorData](onnxFilePath string, inputNames,
|
||||
outputNames []string, inputs, outputs []*Tensor[T]) (*Session[T], error) {
|
||||
fileContent, e := os.ReadFile(onnxFilePath)
|
||||
if e != nil {
|
||||
return nil, fmt.Errorf("Error reading %s: %w", onnxFilePath, e)
|
||||
}
|
||||
|
||||
toReturn, e := NewSessionWithONNXData[T](fileContent, inputNames,
|
||||
outputNames, inputs, outputs)
|
||||
if e != nil {
|
||||
return nil, fmt.Errorf("Error creating session from %s: %w",
|
||||
onnxFilePath, e)
|
||||
}
|
||||
return toReturn, nil
|
||||
}
|
||||
|
||||
// DEPRECATED: See the notes on Session[T]. Use NewDynamicAdvancedSession
|
||||
// instead.
|
||||
func NewDynamicSession[in TensorData, out TensorData](onnxFilePath string,
|
||||
inputNames, outputNames []string) (*DynamicSession[in, out], error) {
|
||||
fileContent, e := os.ReadFile(onnxFilePath)
|
||||
if e != nil {
|
||||
return nil, fmt.Errorf("Error reading %s: %w", onnxFilePath, e)
|
||||
}
|
||||
|
||||
toReturn, e := NewDynamicSessionWithONNXData[in, out](fileContent,
|
||||
inputNames, outputNames)
|
||||
if e != nil {
|
||||
return nil, fmt.Errorf("Error creating session from %s: %w",
|
||||
onnxFilePath, e)
|
||||
}
|
||||
return toReturn, nil
|
||||
}
|
||||
|
||||
func (s *Session[_]) Destroy() error {
|
||||
return s.s.Destroy()
|
||||
}
|
||||
|
||||
func (s *DynamicSession[_, _]) Destroy() error {
|
||||
return s.s.Destroy()
|
||||
}
|
||||
|
||||
func (s *Session[T]) Run() error {
|
||||
return s.s.Run()
|
||||
}
|
||||
|
||||
func (s *DynamicSession[in, out]) Run(inputs []*Tensor[in],
|
||||
outputs []*Tensor[out]) error {
|
||||
if len(inputs) != len(s.s.s.inputNames) {
|
||||
return fmt.Errorf("The session specified %d input names, but Run() "+
|
||||
"was called with %d input tensors", len(s.s.s.inputNames),
|
||||
len(inputs))
|
||||
}
|
||||
if len(outputs) != len(s.s.s.outputNames) {
|
||||
return fmt.Errorf("The session specified %d output names, but Run() "+
|
||||
"was called with %d output tensors", len(s.s.s.outputNames),
|
||||
len(outputs))
|
||||
}
|
||||
inputValues := make([]*C.OrtValue, len(inputs))
|
||||
for i, v := range inputs {
|
||||
inputValues[i] = v.GetInternals().ortValue
|
||||
}
|
||||
outputValues := make([]*C.OrtValue, len(outputs))
|
||||
for i, v := range outputs {
|
||||
outputValues[i] = v.GetInternals().ortValue
|
||||
}
|
||||
|
||||
status := C.RunOrtSession(s.s.s.ortSession, &inputValues[0],
|
||||
&s.s.s.inputNames[0], C.int(len(inputs)), &outputValues[0],
|
||||
&s.s.s.outputNames[0], C.int(len(outputs)))
|
||||
if status != nil {
|
||||
return fmt.Errorf("Error running network: %w", statusToError(status))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// This type alias is included to avoid breaking older code, where the inputs
|
||||
// and outputs to session.Run() were ArbitraryTensors rather than Values.
|
||||
type ArbitraryTensor = Value
|
||||
|
||||
// As with the ArbitraryTensor type, this type alias only exists to facilitate
|
||||
// renaming an old type without breaking existing code.
|
||||
type TensorInternalData = ValueInternalData
|
||||
|
||||
var TrainingAPIRemovedError error = fmt.Errorf("Support for the training " +
|
||||
"API has been removed from onnxruntime_go following its deprecation in " +
|
||||
"onnxruntime versions 1.19.2 and later. The last revision of " +
|
||||
"onnxruntime_go supporting the training API is version v1.12.1")
|
||||
|
||||
// Support for TrainingSessions has been removed from onnxruntime_go following
|
||||
// the deprecation of the training API in onnxruntime 1.20.0.
|
||||
type TrainingSession struct{}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) ExportModel(path string, outputNames []string) error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) SaveCheckpoint(path string,
|
||||
saveOptimizerState bool) error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) Destroy() error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) TrainStep() error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) OptimizerStep() error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns TrainingAPIRemovedError.
|
||||
func (s *TrainingSession) LazyResetGrad() error {
|
||||
return TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Support for TrainingInputOutputNames has been removed from onnxruntime_go
|
||||
// following the deprecation of the training API in onnxruntime 1.20.0.
|
||||
type TrainingInputOutputNames struct {
|
||||
TrainingInputNames []string
|
||||
EvalInputNames []string
|
||||
TrainingOutputNames []string
|
||||
EvalOutputNames []string
|
||||
}
|
||||
|
||||
// Always returns (nil, TrainingAPIRemovedError).
|
||||
func GetInputOutputNames(checkpointStatePath string, trainingModelPath string,
|
||||
evalModelPath string) (*TrainingInputOutputNames, error) {
|
||||
return nil, TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns false.
|
||||
func IsTrainingSupported() bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// Always returns (nil, TrainingAPIRemovedError).
|
||||
func NewTrainingSessionWithOnnxData(checkpointData, trainingData, evalData,
|
||||
optimizerData []byte, inputs, outputs []Value,
|
||||
options *SessionOptions) (*TrainingSession, error) {
|
||||
return nil, TrainingAPIRemovedError
|
||||
}
|
||||
|
||||
// Always returns (nil, TrainingAPIRemovedError).
|
||||
func NewTrainingSession(checkpointStatePath, trainingModelPath, evalModelPath,
|
||||
optimizerModelPath string, inputs, outputs []Value,
|
||||
options *SessionOptions) (*TrainingSession, error) {
|
||||
return nil, TrainingAPIRemovedError
|
||||
}
|
||||
+8574
File diff suppressed because it is too large
Load Diff
+2972
File diff suppressed because it is too large
Load Diff
+3368
File diff suppressed because it is too large
Load Diff
+655
@@ -0,0 +1,655 @@
|
||||
#include "onnxruntime_wrapper.h"
|
||||
|
||||
static const OrtApi *ort_api = NULL;
|
||||
static const char *ORT_VERSION = NULL;
|
||||
|
||||
static AppendCoreMLProviderFn append_coreml_provider_fn = NULL;
|
||||
|
||||
// The dml_provider_factory.h header for using DirectML is annoying to include
|
||||
// here for a couple reasons:
|
||||
// - It contains C++
|
||||
// - It includes d3d12.h and DirectML.h, both of which may be hard to set up
|
||||
// under mingw
|
||||
// Fortunately, the basic AppendExecutionProvider_DML function from the
|
||||
// OrtDmlApi struct does not rely on any of these things, but we still need the
|
||||
// struct definition itself. Obviously, copying it here is not perfect, and
|
||||
// we'll need to keep an eye on it to make sure it doesn't change between
|
||||
// updates. Most importantly, we need to make sure that the one function we
|
||||
// care about remains at the same place in the struct. Since it's first,
|
||||
// hopefully it's unlikely to change.
|
||||
typedef OrtStatus* (*AppendDirectMLProviderFn)(OrtSessionOptions*, int);
|
||||
typedef struct {
|
||||
AppendDirectMLProviderFn SessionOptionsAppendExecutionProvider_DML;
|
||||
// All of these functions pointers should be irrelevant (and they depend on
|
||||
// other definitions from dml_provider_factory.h), but I'll copy them here
|
||||
// regardless as plain void*s. GetExecutionProviderApi shouldn't write to
|
||||
// this struct anyway, as it only provides a const pointer to it.
|
||||
void *SessionOptionsAppendExecutionProvider_DML1;
|
||||
void *CreateGPUAllocationFromD3DResource;
|
||||
void *FreeGPUAllocation;
|
||||
void *GetD3D12ResourceFromAllocation;
|
||||
void *SessionOptionsAppendExecutionProvider_DML2;
|
||||
void *GetDMLDevice;
|
||||
void *GetDMLCommandQueue;
|
||||
} DummyOrtDMLAPI;
|
||||
|
||||
int SetAPIFromBase(OrtApiBase *api_base) {
|
||||
if (!api_base) return 1;
|
||||
ort_api = api_base->GetApi(ORT_API_VERSION);
|
||||
ORT_VERSION = api_base->GetVersionString();
|
||||
if (!ort_api) return 2;
|
||||
return 0;
|
||||
}
|
||||
|
||||
const char *GetVersion() {
|
||||
return ORT_VERSION;
|
||||
}
|
||||
|
||||
void SetCoreMLProviderFunctionPointer(void *ptr) {
|
||||
append_coreml_provider_fn = (AppendCoreMLProviderFn) ptr;
|
||||
}
|
||||
|
||||
void ReleaseOrtStatus(OrtStatus *status) {
|
||||
ort_api->ReleaseStatus(status);
|
||||
}
|
||||
|
||||
OrtStatus *CreateOrtEnv(char *name, OrtEnv **env) {
|
||||
return ort_api->CreateEnv(ORT_LOGGING_LEVEL_ERROR, name, env);
|
||||
}
|
||||
|
||||
OrtStatus *UpdateEnvWithCustomLogLevel(OrtEnv *ort_env,
|
||||
OrtLoggingLevel log_severity_level) {
|
||||
return ort_api->UpdateEnvWithCustomLogLevel(ort_env, log_severity_level);
|
||||
}
|
||||
|
||||
OrtStatus *DisableTelemetry(OrtEnv *env) {
|
||||
return ort_api->DisableTelemetryEvents(env);
|
||||
}
|
||||
|
||||
OrtStatus *EnableTelemetry(OrtEnv *env) {
|
||||
return ort_api->EnableTelemetryEvents(env);
|
||||
}
|
||||
|
||||
void ReleaseOrtEnv(OrtEnv *env) {
|
||||
ort_api->ReleaseEnv(env);
|
||||
}
|
||||
|
||||
OrtStatus *CreateOrtMemoryInfo(OrtMemoryInfo **mem_info) {
|
||||
return ort_api->CreateCpuMemoryInfo(OrtArenaAllocator, OrtMemTypeDefault,
|
||||
mem_info);
|
||||
}
|
||||
|
||||
void ReleaseOrtMemoryInfo(OrtMemoryInfo *info) {
|
||||
ort_api->ReleaseMemoryInfo(info);
|
||||
}
|
||||
|
||||
const char *GetErrorMessage(OrtStatus *status) {
|
||||
if (!status) return "No error (NULL status)";
|
||||
return ort_api->GetErrorMessage(status);
|
||||
}
|
||||
|
||||
OrtStatus *CreateSessionOptions(OrtSessionOptions **o) {
|
||||
return ort_api->CreateSessionOptions(o);
|
||||
}
|
||||
|
||||
void ReleaseSessionOptions(OrtSessionOptions *o) {
|
||||
ort_api->ReleaseSessionOptions(o);
|
||||
}
|
||||
|
||||
OrtStatus *SetSessionExecutionMode(OrtSessionOptions *o, int new_mode) {
|
||||
return ort_api->SetSessionExecutionMode(o, new_mode);
|
||||
}
|
||||
|
||||
OrtStatus *SetSessionGraphOptimizationLevel(OrtSessionOptions *o, int level) {
|
||||
return ort_api->SetSessionGraphOptimizationLevel(o, level);
|
||||
}
|
||||
|
||||
OrtStatus *SetSessionLogSeverityLevel(OrtSessionOptions *o, int level) {
|
||||
return ort_api->SetSessionLogSeverityLevel(o, level);
|
||||
}
|
||||
|
||||
OrtStatus *AddSessionConfigEntry(OrtSessionOptions *o, char *key,
|
||||
char *value) {
|
||||
return ort_api->AddSessionConfigEntry(o, key, value);
|
||||
}
|
||||
|
||||
OrtStatus *HasSessionConfigEntry(OrtSessionOptions *o, char *key,
|
||||
int *result) {
|
||||
return ort_api->HasSessionConfigEntry(o, key, result);
|
||||
}
|
||||
|
||||
// Wraps ort_api->GetSessionConfigEntry
|
||||
OrtStatus *GetSessionConfigEntry(OrtSessionOptions *o, char *key, char *result,
|
||||
size_t *required_size) {
|
||||
return ort_api->GetSessionConfigEntry(o, key, result, required_size);
|
||||
}
|
||||
|
||||
OrtStatus *SetIntraOpNumThreads(OrtSessionOptions *o, int n) {
|
||||
return ort_api->SetIntraOpNumThreads(o, n);
|
||||
}
|
||||
|
||||
OrtStatus *SetInterOpNumThreads(OrtSessionOptions *o, int n) {
|
||||
return ort_api->SetInterOpNumThreads(o, n);
|
||||
}
|
||||
|
||||
OrtStatus *SetCpuMemArena(OrtSessionOptions *o, int use_arena){
|
||||
if (use_arena)
|
||||
return ort_api->EnableCpuMemArena(o);
|
||||
return ort_api->DisableCpuMemArena(o);
|
||||
}
|
||||
|
||||
OrtStatus *SetMemPattern(OrtSessionOptions *o, int use_mem_pattern){
|
||||
if (use_mem_pattern)
|
||||
return ort_api->EnableMemPattern(o);
|
||||
return ort_api->DisableMemPattern(o);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderCUDAV2(OrtSessionOptions *o,
|
||||
OrtCUDAProviderOptionsV2 *cuda_options) {
|
||||
return ort_api->SessionOptionsAppendExecutionProvider_CUDA_V2(o,
|
||||
cuda_options);
|
||||
}
|
||||
|
||||
OrtStatus *CreateCUDAProviderOptions(OrtCUDAProviderOptionsV2 **o) {
|
||||
return ort_api->CreateCUDAProviderOptions(o);
|
||||
}
|
||||
|
||||
void ReleaseCUDAProviderOptions(OrtCUDAProviderOptionsV2 *o) {
|
||||
ort_api->ReleaseCUDAProviderOptions(o);
|
||||
}
|
||||
|
||||
OrtStatus *UpdateCUDAProviderOptions(OrtCUDAProviderOptionsV2 *o,
|
||||
const char **keys, const char **values, int num_keys) {
|
||||
return ort_api->UpdateCUDAProviderOptions(o, keys, values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *CreateTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 **o) {
|
||||
return ort_api->CreateTensorRTProviderOptions(o);
|
||||
}
|
||||
|
||||
void ReleaseTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 *o) {
|
||||
ort_api->ReleaseTensorRTProviderOptions(o);
|
||||
}
|
||||
|
||||
OrtStatus *UpdateTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 *o,
|
||||
const char **keys, const char **values, int num_keys) {
|
||||
return ort_api->UpdateTensorRTProviderOptions(o, keys, values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderTensorRTV2(OrtSessionOptions *o,
|
||||
OrtTensorRTProviderOptionsV2 *tensor_rt_options) {
|
||||
return ort_api->SessionOptionsAppendExecutionProvider_TensorRT_V2(o,
|
||||
tensor_rt_options);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderCoreML(OrtSessionOptions *o,
|
||||
uint32_t flags) {
|
||||
if (!append_coreml_provider_fn) {
|
||||
return ort_api->CreateStatus(ORT_NOT_IMPLEMENTED, "Your platform or "
|
||||
"onnxruntime library does not support CoreML");
|
||||
}
|
||||
return append_coreml_provider_fn(o, flags);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderCoreMLV2(OrtSessionOptions *o,
|
||||
const char **keys, const char **values, size_t num_options) {
|
||||
if (!append_coreml_provider_fn) {
|
||||
return ort_api->CreateStatus(ORT_NOT_IMPLEMENTED, "Your platform or "
|
||||
"onnxruntime library does not support CoreML");
|
||||
}
|
||||
return ort_api->SessionOptionsAppendExecutionProvider(o, "CoreML", keys, values, num_options);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderDirectML(OrtSessionOptions *o,
|
||||
int device_id) {
|
||||
DummyOrtDMLAPI *dml_api = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetExecutionProviderApi("DML", ORT_API_VERSION,
|
||||
(const void **) (&dml_api));
|
||||
if (status) return status;
|
||||
status = dml_api->SessionOptionsAppendExecutionProvider_DML(o, device_id);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderOpenVINOV2(OrtSessionOptions *o,
|
||||
const char **keys, const char **values, int num_keys) {
|
||||
return ort_api->SessionOptionsAppendExecutionProvider_OpenVINO_V2(o, keys,
|
||||
values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProvider(OrtSessionOptions *o,
|
||||
const char *provider_name, const char **keys, const char **values,
|
||||
int num_keys) {
|
||||
return ort_api->SessionOptionsAppendExecutionProvider(o, provider_name,
|
||||
keys, values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *SetOptimizedModelFilePath(OrtSessionOptions *o,
|
||||
char *path) {
|
||||
return ort_api->SetOptimizedModelFilePath(o, (const ORTCHAR_T*) path);
|
||||
}
|
||||
|
||||
OrtStatus *EnableProfiling(OrtSessionOptions *o, char *path) {
|
||||
return ort_api->EnableProfiling(o, (const ORTCHAR_T*) path);
|
||||
}
|
||||
|
||||
OrtStatus *DisableProfiling(OrtSessionOptions *o) {
|
||||
return ort_api->DisableProfiling(o);
|
||||
}
|
||||
|
||||
OrtStatus *RegisterExecutionProviderLibrary(OrtEnv *env,
|
||||
const char *registration_name, char *path) {
|
||||
return ort_api->RegisterExecutionProviderLibrary(env, registration_name,
|
||||
(const ORTCHAR_T *) path);
|
||||
}
|
||||
|
||||
OrtStatus *UnregisterExecutionProviderLibrary(OrtEnv *env,
|
||||
const char *registration_name) {
|
||||
return ort_api->UnregisterExecutionProviderLibrary(env, registration_name);
|
||||
}
|
||||
|
||||
OrtStatus *RegisterCustomOpsLibraryV2(
|
||||
OrtSessionOptions *o,
|
||||
const char *library_path) {
|
||||
return ort_api->RegisterCustomOpsLibrary_V2(o, library_path);
|
||||
}
|
||||
|
||||
OrtStatus *GetEpDevices(OrtEnv *env,
|
||||
const OrtEpDevice * const **out_devices, size_t *out_count) {
|
||||
return ort_api->GetEpDevices(env, out_devices, out_count);
|
||||
}
|
||||
|
||||
const char *EpDeviceEpName(const OrtEpDevice *device) {
|
||||
return ort_api->EpDevice_EpName(device);
|
||||
}
|
||||
|
||||
const char *EpDeviceEpVendor(const OrtEpDevice *device) {
|
||||
return ort_api->EpDevice_EpVendor(device);
|
||||
}
|
||||
|
||||
OrtStatus *AppendExecutionProviderV2(OrtSessionOptions *o, OrtEnv *env,
|
||||
const OrtEpDevice * const *ep_devices, size_t num_ep_devices,
|
||||
const char **keys, const char **values, size_t num_keys) {
|
||||
return ort_api->SessionOptionsAppendExecutionProvider_V2(o, env, ep_devices,
|
||||
num_ep_devices, keys, values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *CreateArenaCfg(size_t max_mem, int arena_extend_strategy,
|
||||
int initial_chunk_size_bytes, int max_dead_bytes_per_chunk,
|
||||
OrtArenaCfg **out) {
|
||||
return ort_api->CreateArenaCfg(max_mem, arena_extend_strategy,
|
||||
initial_chunk_size_bytes, max_dead_bytes_per_chunk, out);
|
||||
}
|
||||
|
||||
OrtStatus *CreateArenaCfgV2(const char *const *arena_config_keys,
|
||||
const size_t *arena_config_values, size_t num_keys, OrtArenaCfg **out) {
|
||||
return ort_api->CreateArenaCfgV2(arena_config_keys, arena_config_values,
|
||||
num_keys, out);
|
||||
}
|
||||
|
||||
void ReleaseArenaCfg(OrtArenaCfg *ptr) {
|
||||
ort_api->ReleaseArenaCfg(ptr);
|
||||
}
|
||||
|
||||
OrtStatus *CreateAndRegisterAllocator(OrtEnv *env,
|
||||
const OrtMemoryInfo *mem_info, const OrtArenaCfg *arena_cfg) {
|
||||
return ort_api->CreateAndRegisterAllocator(env, mem_info, arena_cfg);
|
||||
}
|
||||
|
||||
OrtStatus *CreateAndRegisterAllocatorV2(OrtEnv *env,
|
||||
const char *provider_type, const OrtMemoryInfo *mem_info,
|
||||
const OrtArenaCfg *arena_cfg, const char *const *provider_options_keys,
|
||||
const char *const *provider_options_values, size_t num_keys) {
|
||||
return ort_api->CreateAndRegisterAllocatorV2(env, provider_type, mem_info,
|
||||
arena_cfg, provider_options_keys, provider_options_values, num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *RegisterAllocator(OrtEnv *env, OrtAllocator *allocator) {
|
||||
return ort_api->RegisterAllocator(env, allocator);
|
||||
}
|
||||
|
||||
OrtStatus *UnregisterAllocator(OrtEnv *env, const OrtMemoryInfo *mem_info) {
|
||||
return ort_api->UnregisterAllocator(env, mem_info);
|
||||
}
|
||||
|
||||
OrtStatus *CreateSession(void *model_data, size_t model_data_length,
|
||||
OrtEnv *env, OrtSession **out, OrtSessionOptions *options) {
|
||||
OrtStatus *status = NULL;
|
||||
int default_options = 0;
|
||||
if (!options) {
|
||||
default_options = 1;
|
||||
status = ort_api->CreateSessionOptions(&options);
|
||||
if (status) return status;
|
||||
}
|
||||
status = ort_api->CreateSessionFromArray(env, model_data, model_data_length,
|
||||
options, out);
|
||||
if (default_options) {
|
||||
// If we created a default, empty, options struct, we don't need to keep it
|
||||
// after creating the session.
|
||||
ort_api->ReleaseSessionOptions(options);
|
||||
}
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *CreateSessionFromFile(char *model_path, OrtEnv *env,
|
||||
OrtSession **out, OrtSessionOptions *options) {
|
||||
// Nearly identical to CreateSession, except invokes ort_api->CreateSession
|
||||
// rather than ort_api->CreateSessionFromArray.
|
||||
OrtStatus *status = NULL;
|
||||
int default_options = 0;
|
||||
if (!options) {
|
||||
default_options = 1;
|
||||
status = ort_api->CreateSessionOptions(&options);
|
||||
if (status) return status;
|
||||
}
|
||||
status = ort_api->CreateSession(env, (const ORTCHAR_T*) model_path, options,
|
||||
out);
|
||||
if (default_options) ort_api->ReleaseSessionOptions(options);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *RunOrtSession(OrtSession *session,
|
||||
OrtValue **inputs, char **input_names, int input_count,
|
||||
OrtValue **outputs, char **output_names, int output_count) {
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->Run(session, NULL, (const char* const*) input_names,
|
||||
(const OrtValue* const*) inputs, input_count,
|
||||
(const char* const*) output_names, output_count, outputs);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *RunOrtSessionWithOptions(OrtSession *session,
|
||||
OrtValue **inputs, char **input_names, int input_count,
|
||||
OrtValue **outputs, char **output_names, int output_count,
|
||||
OrtRunOptions *run_options) {
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->Run(session, run_options, (const char* const*) input_names,
|
||||
(const OrtValue* const*) inputs, input_count,
|
||||
(const char* const*) output_names, output_count, outputs);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *CreateRunOptions(OrtRunOptions **o) {
|
||||
return ort_api->CreateRunOptions(o);
|
||||
}
|
||||
|
||||
void ReleaseRunOptions(OrtRunOptions *o) {
|
||||
ort_api->ReleaseRunOptions(o);
|
||||
}
|
||||
|
||||
OrtStatus *RunOptionsSetTerminate(OrtRunOptions *o) {
|
||||
return ort_api->RunOptionsSetTerminate(o);
|
||||
}
|
||||
|
||||
OrtStatus *RunOptionsUnsetTerminate(OrtRunOptions *o) {
|
||||
return ort_api->RunOptionsUnsetTerminate(o);
|
||||
}
|
||||
|
||||
OrtStatus *RunSessionWithBinding(OrtSession *session, OrtIoBinding *b) {
|
||||
return ort_api->RunWithBinding(session, NULL, b);
|
||||
}
|
||||
|
||||
void ReleaseOrtSession(OrtSession *session) {
|
||||
ort_api->ReleaseSession(session);
|
||||
}
|
||||
|
||||
OrtStatus *CreateIoBinding(OrtSession *session, OrtIoBinding **out) {
|
||||
return ort_api->CreateIoBinding(session, out);
|
||||
}
|
||||
|
||||
void ReleaseIoBinding(OrtIoBinding *b) {
|
||||
ort_api->ReleaseIoBinding(b);
|
||||
}
|
||||
|
||||
OrtStatus *BindInput(OrtIoBinding *b, char *name, OrtValue *value) {
|
||||
return ort_api->BindInput(b, name, value);
|
||||
}
|
||||
|
||||
OrtStatus *BindOutput(OrtIoBinding *b, char *name, OrtValue *value) {
|
||||
return ort_api->BindOutput(b, name, value);
|
||||
}
|
||||
|
||||
OrtStatus *GetBoundOutputNames(OrtIoBinding *b, char **buffer,
|
||||
size_t **lengths, size_t *count) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->GetBoundOutputNames(b, allocator, buffer, lengths, count);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetInputCount(OrtSession *session, size_t *result) {
|
||||
return ort_api->SessionGetInputCount(session, result);
|
||||
}
|
||||
|
||||
OrtStatus *GetBoundOutputValues(OrtIoBinding *b, OrtValue ***buffer,
|
||||
size_t *count) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->GetBoundOutputValues(b, allocator, buffer, count);
|
||||
}
|
||||
|
||||
void ClearBoundInputs(OrtIoBinding *b) {
|
||||
ort_api->ClearBoundInputs(b);
|
||||
}
|
||||
|
||||
void ClearBoundOutputs(OrtIoBinding *b) {
|
||||
ort_api->ClearBoundOutputs(b);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetOutputCount(OrtSession *session, size_t *result) {
|
||||
return ort_api->SessionGetOutputCount(session, result);
|
||||
}
|
||||
|
||||
void ReleaseOrtValue(OrtValue *value) {
|
||||
ort_api->ReleaseValue(value);
|
||||
}
|
||||
|
||||
OrtStatus *CreateOrtTensorWithShape(void *data, size_t data_size,
|
||||
int64_t *shape, int64_t shape_size, OrtMemoryInfo *mem_info,
|
||||
ONNXTensorElementDataType dtype, OrtValue **out) {
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->CreateTensorWithDataAsOrtValue(mem_info, data, data_size,
|
||||
shape, shape_size, dtype, out);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *CreateTensorAsOrtValue(int64_t *shape, int64_t shape_size,
|
||||
ONNXTensorElementDataType dtype, OrtValue **out) {
|
||||
OrtStatus *status = NULL;
|
||||
OrtAllocator *allocator = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
status = ort_api->CreateTensorAsOrtValue(allocator, shape, shape_size, dtype,
|
||||
out);
|
||||
return status;
|
||||
}
|
||||
|
||||
OrtStatus *GetTensorTypeAndShape(const OrtValue *value, OrtTensorTypeAndShapeInfo **out) {
|
||||
return ort_api->GetTensorTypeAndShape(value, out);
|
||||
}
|
||||
|
||||
OrtStatus *GetDimensionsCount(const OrtTensorTypeAndShapeInfo *info, size_t *out) {
|
||||
return ort_api->GetDimensionsCount(info, out);
|
||||
}
|
||||
|
||||
OrtStatus *GetDimensions(const OrtTensorTypeAndShapeInfo *info, int64_t *dim_values, size_t dim_values_length) {
|
||||
return ort_api->GetDimensions(info, dim_values, dim_values_length);
|
||||
}
|
||||
|
||||
OrtStatus *GetTensorElementType(const OrtTensorTypeAndShapeInfo *info, enum ONNXTensorElementDataType *out) {
|
||||
return ort_api->GetTensorElementType(info, out);
|
||||
}
|
||||
|
||||
void ReleaseTensorTypeAndShapeInfo(OrtTensorTypeAndShapeInfo *input) {
|
||||
ort_api->ReleaseTensorTypeAndShapeInfo(input);
|
||||
}
|
||||
|
||||
OrtStatus *GetTensorMutableData(OrtValue *value, void **out) {
|
||||
return ort_api->GetTensorMutableData(value, out);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetInputName(OrtSession *session, size_t i, char **name) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->SessionGetInputName(session, i, allocator, name);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetOutputName(OrtSession *session, size_t i, char **name) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->SessionGetOutputName(session, i, allocator, name);
|
||||
}
|
||||
|
||||
OrtStatus *FreeWithDefaultORTAllocator(void *to_free) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->AllocatorFree(allocator, to_free);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetInputTypeInfo(OrtSession *session, size_t i,
|
||||
OrtTypeInfo **out) {
|
||||
return ort_api->SessionGetInputTypeInfo(session, i, out);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetOutputTypeInfo(OrtSession *session, size_t i,
|
||||
OrtTypeInfo **out) {
|
||||
return ort_api->SessionGetOutputTypeInfo(session, i, out);
|
||||
}
|
||||
|
||||
void ReleaseTypeInfo(OrtTypeInfo *o) {
|
||||
ort_api->ReleaseTypeInfo(o);
|
||||
}
|
||||
|
||||
OrtStatus *GetONNXTypeFromTypeInfo(OrtTypeInfo *info, enum ONNXType *out) {
|
||||
return ort_api->GetOnnxTypeFromTypeInfo(info, out);
|
||||
}
|
||||
|
||||
OrtStatus *CastTypeInfoToTensorInfo(OrtTypeInfo *type_info,
|
||||
OrtTensorTypeAndShapeInfo **out) {
|
||||
return ort_api->CastTypeInfoToTensorInfo(type_info,
|
||||
(const OrtTensorTypeAndShapeInfo **) out);
|
||||
}
|
||||
|
||||
OrtStatus *SessionGetModelMetadata(OrtSession *s, OrtModelMetadata **m) {
|
||||
return ort_api->SessionGetModelMetadata(s, m);
|
||||
}
|
||||
|
||||
void ReleaseModelMetadata(OrtModelMetadata *m) {
|
||||
return ort_api->ReleaseModelMetadata(m);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetProducerName(OrtModelMetadata *m, char **name) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataGetProducerName(m, allocator, name);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetGraphName(OrtModelMetadata *m, char **name) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataGetGraphName(m, allocator, name);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetDomain(OrtModelMetadata *m, char **domain) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataGetDomain(m, allocator, domain);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetDescription(OrtModelMetadata *m, char **desc) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataGetDescription(m, allocator, desc);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataLookupCustomMetadataMap(OrtModelMetadata *m, char *key,
|
||||
char **value) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataLookupCustomMetadataMap(m, allocator, key,
|
||||
value);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetCustomMetadataMapKeys(OrtModelMetadata *m,
|
||||
char ***keys, int64_t *num_keys) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->ModelMetadataGetCustomMetadataMapKeys(m, allocator, keys,
|
||||
num_keys);
|
||||
}
|
||||
|
||||
OrtStatus *ModelMetadataGetVersion(OrtModelMetadata *m, int64_t *version) {
|
||||
return ort_api->ModelMetadataGetVersion(m, version);
|
||||
}
|
||||
|
||||
OrtStatus *GetValue(OrtValue *container, int index, OrtValue **dst) {
|
||||
OrtAllocator *allocator = NULL;
|
||||
OrtStatus *status = NULL;
|
||||
status = ort_api->GetAllocatorWithDefaultOptions(&allocator);
|
||||
if (status) return status;
|
||||
return ort_api->GetValue(container, index, allocator, dst);
|
||||
}
|
||||
|
||||
OrtStatus *GetValueType(OrtValue *v, enum ONNXType *out) {
|
||||
return ort_api->GetValueType(v, out);
|
||||
}
|
||||
|
||||
OrtStatus *GetValueCount(OrtValue *v, size_t *out) {
|
||||
return ort_api->GetValueCount(v, out);
|
||||
}
|
||||
|
||||
OrtStatus *CreateOrtValue(OrtValue **in, size_t num_values,
|
||||
enum ONNXType value_type, OrtValue **out) {
|
||||
return ort_api->CreateValue((const OrtValue* const*) in, num_values,
|
||||
value_type, out);
|
||||
}
|
||||
|
||||
OrtStatus *FillStringTensor(OrtValue *v, char **strings, size_t num_strings) {
|
||||
return ort_api->FillStringTensor(v, (const char* const*) strings,
|
||||
num_strings);
|
||||
}
|
||||
|
||||
OrtStatus *GetStringTensorDataLength(OrtValue *v, size_t *length) {
|
||||
return ort_api->GetStringTensorDataLength(v, length);
|
||||
}
|
||||
|
||||
OrtStatus *GetStringTensorContent(OrtValue *v, void *data_buffer,
|
||||
size_t data_size, size_t *offsets_buffer, size_t offsets_length) {
|
||||
return ort_api->GetStringTensorContent(v, data_buffer, data_size,
|
||||
offsets_buffer, offsets_length);
|
||||
}
|
||||
|
||||
OrtStatus *FillStringTensorElement(OrtValue *v, char *s, size_t index) {
|
||||
return ort_api->FillStringTensorElement(v, s, index);
|
||||
}
|
||||
|
||||
OrtStatus *GetStringTensorElementLength(OrtValue *v, size_t index,
|
||||
size_t *result) {
|
||||
return ort_api->GetStringTensorElementLength(v, index, result);
|
||||
}
|
||||
|
||||
OrtStatus *GetStringTensorElement(OrtValue *v, size_t buffer_length,
|
||||
size_t index, void *buffer) {
|
||||
return ort_api->GetStringTensorElement(v, buffer_length, index, buffer);
|
||||
}
|
||||
+438
@@ -0,0 +1,438 @@
|
||||
#ifndef ONNXRUNTIME_WRAPPER_H
|
||||
#define ONNXRUNTIME_WRAPPER_H
|
||||
|
||||
// We want to always use the unix-like onnxruntime C APIs, even on Windows, so
|
||||
// we need to undefine _WIN32 before including onnxruntime_c_api.h. However,
|
||||
// this requires a careful song-and-dance.
|
||||
|
||||
// First, include these common headers, as they get transitively included by
|
||||
// onnxruntime_c_api.h. We need to include them ourselves, first, so that the
|
||||
// preprocessor will skip them while _WIN32 is undefined.
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <string.h>
|
||||
|
||||
// Next, we actually include the header.
|
||||
#undef _WIN32
|
||||
#include "onnxruntime_c_api.h"
|
||||
|
||||
// ... However, mingw will complain if _WIN32 is *not* defined! So redefine it.
|
||||
#define _WIN32
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
// Used for the OrtSessionOptionsAppendExecutionProvider_CoreML function
|
||||
// pointer on supported systems. Must match the signature in
|
||||
// coreml_provider_factory.h provided along with the onnxruntime releases for
|
||||
// Apple platforms.
|
||||
typedef OrtStatus* (*AppendCoreMLProviderFn)(OrtSessionOptions*, uint32_t);
|
||||
|
||||
// Takes a pointer to the api_base struct in order to obtain the OrtApi
|
||||
// pointer. Intended to be called from Go. Returns nonzero on error.
|
||||
int SetAPIFromBase(OrtApiBase *api_base);
|
||||
|
||||
// Get the version of the Onnxruntime library for logging.
|
||||
const char *GetVersion();
|
||||
|
||||
// OrtSessionOptionsAppendExecutionProvider_CoreML is exported directly from
|
||||
// the Apple .dylib, so we call this function on Apple platforms to set the
|
||||
// function pointer to the correct address. On other platforms, the function
|
||||
// pointer should remain NULL.
|
||||
void SetCoreMLProviderFunctionPointer(void *ptr);
|
||||
|
||||
// Wraps ort_api->ReleaseStatus(status)
|
||||
void ReleaseOrtStatus(OrtStatus *status);
|
||||
|
||||
// Wraps calling ort_api->CreateEnv. Returns a non-NULL status on error.
|
||||
OrtStatus *CreateOrtEnv(char *name, OrtEnv **env);
|
||||
|
||||
// Wraps calling ort_api->UpdateEnvWithCustomLogLevel. Return a non-NULL status
|
||||
// on error.
|
||||
OrtStatus *UpdateEnvWithCustomLogLevel(OrtEnv *ort_env,
|
||||
OrtLoggingLevel log_severity_level);
|
||||
|
||||
// Wraps ort_api->DisableTelemetryEvents. Returns a non-NULL status on error.
|
||||
OrtStatus *DisableTelemetry(OrtEnv *env);
|
||||
|
||||
// Wraps ort_api->EnableTelemetryEvents. Returns a non-NULL status on error.
|
||||
OrtStatus *EnableTelemetry(OrtEnv *env);
|
||||
|
||||
// Wraps ort_api->ReleaseEnv
|
||||
void ReleaseOrtEnv(OrtEnv *env);
|
||||
|
||||
// Wraps ort_api->CreateCpuMemoryInfo with some basic, default settings.
|
||||
OrtStatus *CreateOrtMemoryInfo(OrtMemoryInfo **mem_info);
|
||||
|
||||
// Wraps ort_api->ReleaseMemoryInfo
|
||||
void ReleaseOrtMemoryInfo(OrtMemoryInfo *info);
|
||||
|
||||
// Returns the message associated with the given ORT status.
|
||||
const char *GetErrorMessage(OrtStatus *status);
|
||||
|
||||
// Wraps ort_api->CreateSessionOptions
|
||||
OrtStatus *CreateSessionOptions(OrtSessionOptions **o);
|
||||
|
||||
// Wraps ort_api->ReleaseSessionOptions
|
||||
void ReleaseSessionOptions(OrtSessionOptions *o);
|
||||
|
||||
// Wraps ort_api->SetSessionExecutionMode
|
||||
OrtStatus *SetSessionExecutionMode(OrtSessionOptions *o, int new_mode);
|
||||
|
||||
// Wraps ort_api->SetSessionGraphOptimizationLevel
|
||||
OrtStatus *SetSessionGraphOptimizationLevel(OrtSessionOptions *o, int level);
|
||||
|
||||
// Wraps ort_api->SetSessionLogSeverityLevel
|
||||
OrtStatus *SetSessionLogSeverityLevel(OrtSessionOptions *o, int level);
|
||||
|
||||
// Wraps ort_api->AddSessionConfigEntry
|
||||
OrtStatus *AddSessionConfigEntry(OrtSessionOptions *o, char *key, char *value);
|
||||
|
||||
// Wraps ort_api->HasSessionConfigEntry
|
||||
OrtStatus *HasSessionConfigEntry(OrtSessionOptions *o, char *key, int *result);
|
||||
|
||||
// Wraps ort_api->GetSessionConfigEntry
|
||||
OrtStatus *GetSessionConfigEntry(OrtSessionOptions *o, char *key, char *result,
|
||||
size_t *required_size);
|
||||
|
||||
// Wraps ort_api->SetIntraOpNumThreads
|
||||
OrtStatus *SetIntraOpNumThreads(OrtSessionOptions *o, int n);
|
||||
|
||||
// Wraps ort_api->SetInterOpNumThreads
|
||||
OrtStatus *SetInterOpNumThreads(OrtSessionOptions *o, int n);
|
||||
|
||||
// Wraps ort_api->EnableCpuMemArena & ort_api->DisableCpuMemArena
|
||||
OrtStatus *SetCpuMemArena(OrtSessionOptions *o, int use_arena);
|
||||
|
||||
// Wraps ort_api->EnableMemPattern & ort_api->DisableMemPattern
|
||||
OrtStatus *SetMemPattern(OrtSessionOptions *o, int use_mem_pattern);
|
||||
|
||||
// Wraps ort_api->SessionOptionsAppendExecutionProvider_CUDA_V2
|
||||
OrtStatus *AppendExecutionProviderCUDAV2(OrtSessionOptions *o,
|
||||
OrtCUDAProviderOptionsV2 *cuda_options);
|
||||
|
||||
// Wraps ort_api->CreateCUDAProviderOptions
|
||||
OrtStatus *CreateCUDAProviderOptions(OrtCUDAProviderOptionsV2 **o);
|
||||
|
||||
// Wraps ort_api->ReleaseCUDAProviderOptions
|
||||
void ReleaseCUDAProviderOptions(OrtCUDAProviderOptionsV2 *o);
|
||||
|
||||
// Wraps ort_api->UpdateCUDAProviderOptions
|
||||
OrtStatus *UpdateCUDAProviderOptions(OrtCUDAProviderOptionsV2 *o,
|
||||
const char **keys, const char **values, int num_keys);
|
||||
|
||||
// Wraps ort_api->CreateTensorRTProviderOptions
|
||||
OrtStatus *CreateTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 **o);
|
||||
|
||||
// Wraps ort_api->ReleaseTensorRTProviderOptions
|
||||
void ReleaseTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 *o);
|
||||
|
||||
// Wraps ort_api->UpdateTensorRTProviderOptions
|
||||
OrtStatus *UpdateTensorRTProviderOptions(OrtTensorRTProviderOptionsV2 *o,
|
||||
const char **keys, const char **values, int num_keys);
|
||||
|
||||
// Wraps ort_api->SessionOptionsAppendExecutionProvider_TensorRT_V2
|
||||
OrtStatus *AppendExecutionProviderTensorRTV2(OrtSessionOptions *o,
|
||||
OrtTensorRTProviderOptionsV2 *tensor_rt_options);
|
||||
|
||||
// Wraps OrtSessionOptionsAppendExecutionProvider_CoreML, exported from the
|
||||
// dylib on Apple devices. Safely returns a non-NULL status on other platforms.
|
||||
OrtStatus *AppendExecutionProviderCoreML(OrtSessionOptions *o,
|
||||
uint32_t flags);
|
||||
|
||||
// Wraps OrtApi method SessionOptionsAppendExecutionProvider specifically for CoreML
|
||||
// with the new options-based API while including the check that coreml is supported.
|
||||
OrtStatus *AppendExecutionProviderCoreMLV2(OrtSessionOptions *o,
|
||||
const char **keys, const char **values, size_t num_options);
|
||||
|
||||
// Wraps getting the OrtDmlApi struct and calling
|
||||
// dml_api->SessionOptionsAppendExecutionProvider_DML.
|
||||
OrtStatus *AppendExecutionProviderDirectML(OrtSessionOptions *o,
|
||||
int device_id);
|
||||
|
||||
// Wraps ort_api->AppendExecutionProvider_OpenVINO_V2
|
||||
OrtStatus *AppendExecutionProviderOpenVINOV2(OrtSessionOptions *o,
|
||||
const char **keys, const char **values, int num_keys);
|
||||
|
||||
// Wraps ort_api->SessionOptionsAppendExecutionProvider
|
||||
OrtStatus *AppendExecutionProvider(OrtSessionOptions *o,
|
||||
const char *provider_name, const char **keys, const char **values,
|
||||
int num_keys);
|
||||
|
||||
// Wraps ort_api->SetOptimizedModelFilePath. NOTE: takes an ORTCHAR_T*.
|
||||
OrtStatus *SetOptimizedModelFilePath(OrtSessionOptions *o,
|
||||
char *path);
|
||||
|
||||
// Wraps ort_api->EnableProfiling. NOTE: takes an ORTCHAR_T*.
|
||||
OrtStatus *EnableProfiling(OrtSessionOptions *o, char *path);
|
||||
|
||||
// Wraps ort_api->DisableProfiling.
|
||||
OrtStatus *DisableProfiling(OrtSessionOptions *o);
|
||||
|
||||
// Wraps ort_api->RegisterExecutionProviderLibrary
|
||||
OrtStatus *RegisterExecutionProviderLibrary(OrtEnv *env,
|
||||
const char *registration_name, char *path);
|
||||
|
||||
// Wraps ort_api->RegisterCustomOpsLibraryV2
|
||||
OrtStatus *RegisterCustomOpsLibraryV2(OrtSessionOptions *o,
|
||||
const char *library_name);
|
||||
|
||||
// Wraps ort_api->UnregisterExecutionProviderLibrary
|
||||
OrtStatus *UnregisterExecutionProviderLibrary(OrtEnv *env,
|
||||
const char *registration_name);
|
||||
|
||||
// Wraps ort_api->GetEpDevices. The returned array is owned by the OrtEnv and
|
||||
// must NOT be freed by the caller; the pointers inside it remain valid until
|
||||
// the corresponding plugin library is unregistered or the env is released.
|
||||
OrtStatus *GetEpDevices(OrtEnv *env,
|
||||
const OrtEpDevice * const **out_devices, size_t *out_count);
|
||||
|
||||
// Wraps ort_api->EpDevice_EpName.
|
||||
const char *EpDeviceEpName(const OrtEpDevice *device);
|
||||
|
||||
// Wraps ort_api->EpDevice_EpVendor.
|
||||
const char *EpDeviceEpVendor(const OrtEpDevice *device);
|
||||
|
||||
// Wraps ort_api->SessionOptionsAppendExecutionProvider_V2.
|
||||
OrtStatus *AppendExecutionProviderV2(OrtSessionOptions *o, OrtEnv *env,
|
||||
const OrtEpDevice * const *ep_devices, size_t num_ep_devices,
|
||||
const char **keys, const char **values, size_t num_keys);
|
||||
|
||||
// Wraps ort_api->CreateArenaCfg
|
||||
OrtStatus *CreateArenaCfg(size_t max_mem, int arena_extend_strategy,
|
||||
int initial_chunk_size_bytes, int max_dead_bytes_per_chunk,
|
||||
OrtArenaCfg **out);
|
||||
|
||||
// Wraps ort_api->CreateArenaCfgV2
|
||||
OrtStatus *CreateArenaCfgV2(const char *const *arena_config_keys,
|
||||
const size_t *arena_config_values, size_t num_keys, OrtArenaCfg **out);
|
||||
|
||||
// Wraps ort_api->ReleaseArenaCfg
|
||||
void ReleaseArenaCfg(OrtArenaCfg *ptr);
|
||||
|
||||
// Wraps ort_api->CreateAndRegisterAllocator
|
||||
OrtStatus *CreateAndRegisterAllocator(OrtEnv *env,
|
||||
const OrtMemoryInfo *mem_info, const OrtArenaCfg *arena_cfg);
|
||||
|
||||
// Wraps ort_api->CreateAndRegisterAllocatorV2
|
||||
OrtStatus *CreateAndRegisterAllocatorV2(OrtEnv *env,
|
||||
const char *provider_type, const OrtMemoryInfo *mem_info,
|
||||
const OrtArenaCfg *arena_cfg, const char *const *provider_options_keys,
|
||||
const char *const *provider_options_values, size_t num_keys);
|
||||
|
||||
// Wraps ort_api->RegisterAllocator
|
||||
OrtStatus *RegisterAllocator(OrtEnv *env, OrtAllocator *allocator);
|
||||
|
||||
// Wraps ort_api->UnregisterAllocator
|
||||
OrtStatus *UnregisterAllocator(OrtEnv *env, const OrtMemoryInfo *mem_info);
|
||||
|
||||
// Creates an ORT session using the given model. The given options pointer may
|
||||
// be NULL; if it is, then we'll use default options.
|
||||
OrtStatus *CreateSession(void *model_data, size_t model_data_length,
|
||||
OrtEnv *env, OrtSession **out, OrtSessionOptions *options);
|
||||
|
||||
// Like the CreateSession function, but takes a path to a model rather than a
|
||||
// buffer containing it.
|
||||
OrtStatus *CreateSessionFromFile(char *model_path, OrtEnv *env,
|
||||
OrtSession **out, OrtSessionOptions *options);
|
||||
|
||||
// Runs an ORT session with the given input and output tensors, along with
|
||||
// their names.
|
||||
OrtStatus *RunOrtSession(OrtSession *session,
|
||||
OrtValue **inputs, char **input_names, int input_count,
|
||||
OrtValue **outputs, char **output_names, int output_count);
|
||||
|
||||
// Runs an ORT session with explicit OrtRunOptions.
|
||||
OrtStatus *RunOrtSessionWithOptions(OrtSession *session,
|
||||
OrtValue **inputs, char **input_names, int input_count,
|
||||
OrtValue **outputs, char **output_names, int output_count,
|
||||
OrtRunOptions *run_options);
|
||||
|
||||
// RunOptions helpers
|
||||
// Wraps ort_api->CreateRunOptions
|
||||
OrtStatus *CreateRunOptions(OrtRunOptions **o);
|
||||
// Wraps ort_api->ReleaseRunOptions
|
||||
void ReleaseRunOptions(OrtRunOptions *o);
|
||||
// Wraps ort_api->RunOptionsSetTerminate
|
||||
OrtStatus *RunOptionsSetTerminate(OrtRunOptions *o);
|
||||
// Wraps ort_api->RunOptionsUnsetTerminate
|
||||
OrtStatus *RunOptionsUnsetTerminate(OrtRunOptions *o);
|
||||
|
||||
// Wraps ort_api->RunWithBinding.
|
||||
OrtStatus *RunSessionWithBinding(OrtSession *session, OrtIoBinding *b);
|
||||
|
||||
// Wraps ort_api->ReleaseSession
|
||||
void ReleaseOrtSession(OrtSession *session);
|
||||
|
||||
// Wraps ort_api->CreateIoBinding
|
||||
OrtStatus *CreateIoBinding(OrtSession *session, OrtIoBinding **out);
|
||||
|
||||
// Wraps ort_api->ReleaseIoBinding
|
||||
void ReleaseIoBinding(OrtIoBinding *b);
|
||||
|
||||
// Wraps ort_api->BindInput
|
||||
OrtStatus *BindInput(OrtIoBinding *b, char *name, OrtValue *value);
|
||||
|
||||
// Wraps ort_api->BindOutput
|
||||
OrtStatus *BindOutput(OrtIoBinding *b, char *name, OrtValue *value);
|
||||
|
||||
// Wraps ort_api->GetBoundOutputNames. Uses the default allocator. The caller
|
||||
// must free the buffer and lengths using FreeWithDefaultOrtAllocator after
|
||||
// this is done.
|
||||
OrtStatus *GetBoundOutputNames(OrtIoBinding *b, char **buffer,
|
||||
size_t **lengths, size_t *count);
|
||||
|
||||
// Wraps ort_api->GetBoundOutputNames. Uses the default allocator, and the
|
||||
// caller must free the buffer of values using FreeWithDefaultOrtAllocator.
|
||||
OrtStatus *GetBoundOutputValues(OrtIoBinding *b, OrtValue ***buffer,
|
||||
size_t *count);
|
||||
|
||||
// Wraps ort_api->ClearBoundInputs.
|
||||
void ClearBoundInputs(OrtIoBinding *b);
|
||||
|
||||
// Wraps ort_api->ClearBoundOutputs.
|
||||
void ClearBoundOutputs(OrtIoBinding *b);
|
||||
|
||||
// Wraps ort_api->SessionGetInputCount.
|
||||
OrtStatus *SessionGetInputCount(OrtSession *session, size_t *result);
|
||||
|
||||
// Wraps ort_api->SessionGetOutputCount.
|
||||
OrtStatus *SessionGetOutputCount(OrtSession *session, size_t *result);
|
||||
|
||||
// Used to free OrtValue instances, such as tensors.
|
||||
void ReleaseOrtValue(OrtValue *value);
|
||||
|
||||
// Creates an OrtValue tensor with the given shape, and backed by the user-
|
||||
// supplied data buffer.
|
||||
OrtStatus *CreateOrtTensorWithShape(void *data, size_t data_size,
|
||||
int64_t *shape, int64_t shape_size, OrtMemoryInfo *mem_info,
|
||||
ONNXTensorElementDataType dtype, OrtValue **out);
|
||||
|
||||
// Creates an OrtValue managed by onnxruntime's default allocator rather than
|
||||
// using Go-managed memory. Wraps ort_api->CreateTensorAsOrtValue.
|
||||
OrtStatus *CreateTensorAsOrtValue(int64_t *shape, int64_t shape_size,
|
||||
ONNXTensorElementDataType dtype, OrtValue **out);
|
||||
|
||||
// Wraps ort_api->GetTensorTypeAndShape
|
||||
OrtStatus *GetTensorTypeAndShape(const OrtValue *value,
|
||||
OrtTensorTypeAndShapeInfo **out);
|
||||
|
||||
// Wraps ort_api->GetDimensionsCount
|
||||
OrtStatus *GetDimensionsCount(const OrtTensorTypeAndShapeInfo *info,
|
||||
size_t *out);
|
||||
|
||||
// Wraps ort_api->GetDimensions
|
||||
OrtStatus *GetDimensions(const OrtTensorTypeAndShapeInfo *info,
|
||||
int64_t *dim_values, size_t dim_values_length);
|
||||
|
||||
// Wraps ort_api->GetTensorElementType
|
||||
OrtStatus *GetTensorElementType(const OrtTensorTypeAndShapeInfo *info,
|
||||
enum ONNXTensorElementDataType *out);
|
||||
|
||||
// Wraps ort_api->ReleaseTensorTypeAndShapeInfo
|
||||
void ReleaseTensorTypeAndShapeInfo(OrtTensorTypeAndShapeInfo *input);
|
||||
|
||||
// Wraps ort_api->GetTensorMutableData
|
||||
OrtStatus *GetTensorMutableData(OrtValue *value, void **out);
|
||||
|
||||
// Wraps ort_api->SessionGetInputName, using the default allocator.
|
||||
OrtStatus *SessionGetInputName(OrtSession *session, size_t i, char **name);
|
||||
|
||||
// Wraps ort_api->SessionGetOutputName, using the default allocator.
|
||||
OrtStatus *SessionGetOutputName(OrtSession *session, size_t i, char **name);
|
||||
|
||||
// Frees anything that was allocated using the default ORT allocator.
|
||||
OrtStatus *FreeWithDefaultORTAllocator(void *to_free);
|
||||
|
||||
// Wraps ort_api->SessionGetInputTypeInfo.
|
||||
OrtStatus *SessionGetInputTypeInfo(OrtSession *session, size_t i,
|
||||
OrtTypeInfo **out);
|
||||
|
||||
// Wraps ort_api->SessionGetOutputTypeInfo.
|
||||
OrtStatus *SessionGetOutputTypeInfo(OrtSession *session, size_t i,
|
||||
OrtTypeInfo **out);
|
||||
|
||||
// If the type_info is for a tensor, sets out to the a pointer to the tensor's
|
||||
// NameAndTypeInfo. Do _not_ free the out pointer; it will be freed when
|
||||
// type_info is released.
|
||||
//
|
||||
// Wraps ort_api->CastTypeInfoToTensorInfo.
|
||||
OrtStatus *CastTypeInfoToTensorInfo(OrtTypeInfo *type_info,
|
||||
OrtTensorTypeAndShapeInfo **out);
|
||||
|
||||
// Wraps ort_api->GetOnnxTypeFromTypeInfo.
|
||||
OrtStatus *GetONNXTypeFromTypeInfo(OrtTypeInfo *info, enum ONNXType *out);
|
||||
|
||||
// Wraps ort_api->FreeTypeInfo.
|
||||
void ReleaseTypeInfo(OrtTypeInfo *o);
|
||||
|
||||
// Wraps ort_spi->SessionGetModelMetadata.
|
||||
OrtStatus *SessionGetModelMetadata(OrtSession *s, OrtModelMetadata **out);
|
||||
|
||||
// Wraps ort_api->ReleaseModelMetadata.
|
||||
void ReleaseModelMetadata(OrtModelMetadata *m);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetProducerName, using the default allocator.
|
||||
OrtStatus *ModelMetadataGetProducerName(OrtModelMetadata *m, char **name);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetGraphName, using the default allocator.
|
||||
OrtStatus *ModelMetadataGetGraphName(OrtModelMetadata *m, char **name);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetDomain, using the default allocator.
|
||||
OrtStatus *ModelMetadataGetDomain(OrtModelMetadata *m, char **domain);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetDescription, using the default allocator.
|
||||
OrtStatus *ModelMetadataGetDescription(OrtModelMetadata *m, char **desc);
|
||||
|
||||
// Wraps ort_api->ModelMetadataLookupCustomMetadataMap, using the default
|
||||
// allocator.
|
||||
OrtStatus *ModelMetadataLookupCustomMetadataMap(OrtModelMetadata *m, char *key,
|
||||
char **value);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetCustomMetadataMapKeys, using the default
|
||||
// allocator.
|
||||
OrtStatus *ModelMetadataGetCustomMetadataMapKeys(OrtModelMetadata *m,
|
||||
char ***keys, int64_t *num_keys);
|
||||
|
||||
// Wraps ort_api->ModelMetadataGetVersion.
|
||||
OrtStatus *ModelMetadataGetVersion(OrtModelMetadata *m, int64_t *version);
|
||||
|
||||
// Wraps ort_api->GetValue. Uses the default allocator.
|
||||
OrtStatus *GetValue(OrtValue *container, int index, OrtValue **dst);
|
||||
|
||||
// Wraps ort_api->GetValueType.
|
||||
OrtStatus *GetValueType(OrtValue *v, enum ONNXType *out);
|
||||
|
||||
// Wraps ort_api->GetValueCount.
|
||||
OrtStatus *GetValueCount(OrtValue *v, size_t *out);
|
||||
|
||||
// Wraps ort_api->CreateValue to create a map or a sequence.
|
||||
OrtStatus *CreateOrtValue(OrtValue **in, size_t num_values,
|
||||
enum ONNXType value_type, OrtValue **out);
|
||||
|
||||
// Wraps ort_api->FillStringTensor
|
||||
OrtStatus *FillStringTensor(OrtValue *v, char **strings, size_t num_strings);
|
||||
|
||||
// Wraps ort_api->GetStringTensorDataLength
|
||||
OrtStatus *GetStringTensorDataLength(OrtValue *v, size_t *length);
|
||||
|
||||
// Wraps ort_api->GetStringTensorContent
|
||||
OrtStatus *GetStringTensorContent(OrtValue *v, void *data_buffer,
|
||||
size_t data_size, size_t *offsets_buffer, size_t offsets_length);
|
||||
|
||||
// Wraps ort_api->FillStringTensorElement
|
||||
OrtStatus *FillStringTensorElement(OrtValue *v, char *s, size_t index);
|
||||
|
||||
// Wraps ort_api->GetStringTensorElementLength
|
||||
OrtStatus *GetStringTensorElementLength(OrtValue *v, size_t index,
|
||||
size_t *result);
|
||||
|
||||
// Wraps ort_api->GetStringTensorElement
|
||||
OrtStatus *GetStringTensorElement(OrtValue *v, size_t buffer_length,
|
||||
size_t index, void *buffer);
|
||||
|
||||
#ifdef __cplusplus
|
||||
} // extern "C"
|
||||
#endif
|
||||
#endif // ONNXRUNTIME_WRAPPER_H
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
//go:build !windows
|
||||
|
||||
package onnxruntime_go
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"runtime"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
/*
|
||||
#cgo LDFLAGS: -ldl
|
||||
|
||||
#include <dlfcn.h>
|
||||
#include "onnxruntime_wrapper.h"
|
||||
|
||||
typedef OrtApiBase* (*GetOrtApiBaseFunction)(void);
|
||||
|
||||
// Since Go can't call C function pointers directly, we just use this helper
|
||||
// when calling GetApiBase
|
||||
OrtApiBase *CallGetAPIBaseFunction(void *fn) {
|
||||
OrtApiBase *to_return = ((GetOrtApiBaseFunction) fn)();
|
||||
return to_return;
|
||||
}
|
||||
*/
|
||||
import "C"
|
||||
|
||||
// This file includes the code for loading the onnxruntime and setting up the
|
||||
// environment on non-Windows systems. For now, it has been tested on Linux and
|
||||
// arm64 OSX.
|
||||
|
||||
// This will contain the handle to the onnxruntime shared library if it has
|
||||
// been loaded successfully.
|
||||
var libraryHandle unsafe.Pointer
|
||||
|
||||
func platformCleanup() error {
|
||||
v, e := C.dlclose(libraryHandle)
|
||||
if v != 0 {
|
||||
return fmt.Errorf("Error closing the library: %w", e)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Should only be called on Apple systems; looks up the CoreML provider
|
||||
// function which should only be exported on apple onnxruntime dylib files.
|
||||
func setAppendCoreMLFunctionPointer(libraryHandle unsafe.Pointer) error {
|
||||
// This function name must match the name in coreml_provider_factory.h,
|
||||
// which is provided in the onnxruntime release's include/ directory on for
|
||||
// Apple platforms.
|
||||
fnName := "OrtSessionOptionsAppendExecutionProvider_CoreML"
|
||||
cFunctionName := C.CString(fnName)
|
||||
defer C.free(unsafe.Pointer(cFunctionName))
|
||||
appendCoreMLProviderProc := C.dlsym(libraryHandle, cFunctionName)
|
||||
if appendCoreMLProviderProc == nil {
|
||||
msg := C.GoString(C.dlerror())
|
||||
return fmt.Errorf("Error looking up %s: %s", fnName, msg)
|
||||
}
|
||||
C.SetCoreMLProviderFunctionPointer(appendCoreMLProviderProc)
|
||||
return nil
|
||||
}
|
||||
|
||||
func platformInitializeEnvironment() error {
|
||||
if onnxSharedLibraryPath == "" {
|
||||
onnxSharedLibraryPath = "onnxruntime.so"
|
||||
}
|
||||
cName := C.CString(onnxSharedLibraryPath)
|
||||
defer C.free(unsafe.Pointer(cName))
|
||||
handle := C.dlopen(cName, C.RTLD_LAZY)
|
||||
if handle == nil {
|
||||
msg := C.GoString(C.dlerror())
|
||||
return fmt.Errorf("Error loading ONNX shared library \"%s\": %s",
|
||||
onnxSharedLibraryPath, msg)
|
||||
}
|
||||
cFunctionName := C.CString("OrtGetApiBase")
|
||||
defer C.free(unsafe.Pointer(cFunctionName))
|
||||
getAPIBaseProc := C.dlsym(handle, cFunctionName)
|
||||
if getAPIBaseProc == nil {
|
||||
C.dlclose(handle)
|
||||
msg := C.GoString(C.dlerror())
|
||||
return fmt.Errorf("Error looking up OrtGetApiBase in \"%s\": %s",
|
||||
onnxSharedLibraryPath, msg)
|
||||
}
|
||||
ortAPIBase := C.CallGetAPIBaseFunction(getAPIBaseProc)
|
||||
tmp := C.SetAPIFromBase((*C.OrtApiBase)(unsafe.Pointer(ortAPIBase)))
|
||||
if tmp != 0 {
|
||||
C.dlclose(handle)
|
||||
return fmt.Errorf("Error setting ORT API base: %d", tmp)
|
||||
}
|
||||
if (runtime.GOOS == "darwin") || (runtime.GOOS == "ios") {
|
||||
setAppendCoreMLFunctionPointer(handle)
|
||||
// We'll silently ignore potential errors returned by
|
||||
// setAppendCoreMLFunctionPointer (for now at least). Even though we're
|
||||
// on Apple hardware, it's possible that the user will have compiled
|
||||
// the onnxruntime library from source without CoreML support.
|
||||
// A failure here will only leave the coreml function pointer as NULL
|
||||
// in our C code, which will be detected and result in an error at
|
||||
// runtime.
|
||||
}
|
||||
libraryHandle = handle
|
||||
return nil
|
||||
}
|
||||
|
||||
// Converts the given path to an ORTCHAR_T string, pointed to by a *C.char. The
|
||||
// returned string must be freed using C.free when no longer needed. This
|
||||
// wrapper is used for source compatibility with onnxruntime API functions
|
||||
// requiring paths, which must be UTF-16 on Windows but UTF-8 elsewhere.
|
||||
func createOrtCharString(str string) (*C.char, error) {
|
||||
return C.CString(str), nil
|
||||
}
|
||||
+103
@@ -0,0 +1,103 @@
|
||||
//go:build windows
|
||||
|
||||
package onnxruntime_go
|
||||
|
||||
// This file includes the Windows-specific code for loading the onnxruntime
|
||||
// library and setting up the environment.
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"syscall"
|
||||
"unicode/utf16"
|
||||
"unicode/utf8"
|
||||
"unsafe"
|
||||
)
|
||||
|
||||
// #include "onnxruntime_wrapper.h"
|
||||
import "C"
|
||||
|
||||
// This will contain the handle to the onnxruntime dll if it has been loaded
|
||||
// successfully.
|
||||
var libraryHandle syscall.Handle
|
||||
|
||||
func platformCleanup() error {
|
||||
e := syscall.FreeLibrary(libraryHandle)
|
||||
libraryHandle = 0
|
||||
return e
|
||||
}
|
||||
|
||||
func platformInitializeEnvironment() error {
|
||||
if onnxSharedLibraryPath == "" {
|
||||
onnxSharedLibraryPath = "onnxruntime.dll"
|
||||
}
|
||||
handle, e := syscall.LoadLibrary(onnxSharedLibraryPath)
|
||||
if e != nil {
|
||||
return fmt.Errorf("Error loading ONNX shared library \"%s\": %w",
|
||||
onnxSharedLibraryPath, e)
|
||||
}
|
||||
getApiBaseProc, e := syscall.GetProcAddress(handle, "OrtGetApiBase")
|
||||
if e != nil {
|
||||
syscall.FreeLibrary(handle)
|
||||
return fmt.Errorf("Error finding OrtGetApiBase function in %s: %w",
|
||||
onnxSharedLibraryPath, e)
|
||||
}
|
||||
ortApiBase, _, e := syscall.SyscallN(uintptr(getApiBaseProc), 0)
|
||||
if ortApiBase == 0 {
|
||||
syscall.FreeLibrary(handle)
|
||||
if e != nil {
|
||||
return fmt.Errorf("Error calling OrtGetApiBase: %w", e)
|
||||
} else {
|
||||
return fmt.Errorf("Error calling OrtGetApiBase")
|
||||
}
|
||||
}
|
||||
tmp := C.SetAPIFromBase((*C.OrtApiBase)(unsafe.Pointer(ortApiBase)))
|
||||
if tmp != 0 {
|
||||
syscall.FreeLibrary(handle)
|
||||
return fmt.Errorf("Error setting ORT API base: %d", tmp)
|
||||
}
|
||||
|
||||
libraryHandle = handle
|
||||
return nil
|
||||
}
|
||||
|
||||
// Converts the given string to a UTF-16 string, pointed to by a raw
|
||||
// *C.char. Note that we actually keep ORTCHAR_T defined to char even
|
||||
// on Windows, so do _not_ index into this string from Cgo code and expect to
|
||||
// get correct characters! Instead, this should only be used to obtain pointers
|
||||
// that are passed to onnxruntime windows DLL functions expecting ORTCHAR_T*
|
||||
// args. This is required because we undefine _WIN32 for cgo compatibility when
|
||||
// including onnxruntime_c_api.h, but still interact with a DLL that was
|
||||
// compiled assuming _WIN32 was defined.
|
||||
//
|
||||
// The pointer returned by this function must still be freed using C.free when
|
||||
// no longer needed. This will return an error if the given string contains
|
||||
// non-UTF8 characters.
|
||||
func createOrtCharString(str string) (*C.char, error) {
|
||||
src := []uint8(str)
|
||||
// Assumed common case: the utf16 buffer contains one uint16 per utf8 byte
|
||||
// plus one more for the required null terminator in the C buffer.
|
||||
dst := make([]uint16, 0, len(src)+1)
|
||||
// Convert UTF-8 to UTF-16 by reading each subsequent rune from src and
|
||||
// appending it as UTF-16 to dst.
|
||||
for len(src) > 0 {
|
||||
r, size := utf8.DecodeRune(src)
|
||||
if r == utf8.RuneError {
|
||||
return nil, fmt.Errorf("Invalid UTF-8 rune found in \"%s\"", str)
|
||||
}
|
||||
src = src[size:]
|
||||
dst = utf16.AppendRune(dst, r)
|
||||
}
|
||||
// Make sure dst contains the null terminator. Additionally this will cause
|
||||
// us to return an empty string if the original string was empty.
|
||||
dst = append(dst, 0)
|
||||
|
||||
// Finally, we need to copy dst into a C array for compatibility with
|
||||
// C.CString.
|
||||
toReturn := C.calloc(C.size_t(len(dst)), 2)
|
||||
if toReturn == nil {
|
||||
return nil, fmt.Errorf("Error allocating buffer for the utf16 string")
|
||||
}
|
||||
C.memcpy(toReturn, unsafe.Pointer(&(dst[0])), C.size_t(len(dst))*2)
|
||||
|
||||
return (*C.char)(toReturn), nil
|
||||
}
|
||||
+56
@@ -0,0 +1,56 @@
|
||||
package onnxruntime_go
|
||||
|
||||
// This file contains definitions for the generic tensor data types we support.
|
||||
|
||||
// #include "onnxruntime_wrapper.h"
|
||||
import "C"
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
)
|
||||
|
||||
type FloatData interface {
|
||||
~float32 | ~float64
|
||||
}
|
||||
|
||||
type IntData interface {
|
||||
~int8 | ~uint8 | ~int16 | ~uint16 | ~int32 | ~uint32 | ~int64 | ~uint64
|
||||
}
|
||||
|
||||
// This is used as a type constraint for the generic Tensor type.
|
||||
type TensorData interface {
|
||||
FloatData | IntData | ~bool
|
||||
}
|
||||
|
||||
// Returns the ONNX enum value used to indicate TensorData type T.
|
||||
func GetTensorElementDataType[T TensorData]() C.ONNXTensorElementDataType {
|
||||
// Sadly, we can't do type assertions to get underlying types, so we need
|
||||
// to use reflect here instead.
|
||||
var v T
|
||||
kind := reflect.ValueOf(v).Kind()
|
||||
switch kind {
|
||||
case reflect.Float64:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_DOUBLE
|
||||
case reflect.Float32:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_FLOAT
|
||||
case reflect.Int8:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT8
|
||||
case reflect.Uint8:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT8
|
||||
case reflect.Int16:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT16
|
||||
case reflect.Uint16:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT16
|
||||
case reflect.Int32:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT32
|
||||
case reflect.Uint32:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT32
|
||||
case reflect.Int64:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_INT64
|
||||
case reflect.Uint64:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UINT64
|
||||
case reflect.Bool:
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_BOOL
|
||||
}
|
||||
return C.ONNX_TENSOR_ELEMENT_DATA_TYPE_UNDEFINED
|
||||
}
|
||||
Reference in New Issue
Block a user