Building a Frontend

PlaidML currently supports Keras and ONNX as frontends. This document explains how to add PlaidML as a backend to other machine learning frontends.

Read the Frontend’s Documentation

Most frontends document their backend API. Frontends differ in their requirements of backends, and you may need to adapt these instructions to the needs of the frontend. Reading the frontend documentation will help you implement the necessary features in the expected way.

You may also find unit tests available for running the frontend using PlaidML.

API Overview

The modules plaidml, plaidml.op, and plaidml.tile are of particular use in constructing a frontend. They respectively provide general purpose classes, a suite of commonly used operations, and a library for constructing and composing operations.

Required Functionality

Broadly speaking, to implement a frontend you will need to provide the following:
  • A means of communicating with the execution device, as discussed in Context & Device
  • Implementations of operations requested by the frontend, as discussed in Individual Operations
  • Construction of a computation graph, as discussed in Functions and Operations & Values. The frontend may handle some of this.
  • A supply of input data to a computation graph, execution of that graph, and recovery of its output, as discussed in Functions and Invokers

The frontend’s documentation may describe additional functionality that is also required.

Context & Device

The class Context provides a context for executing code on a device as requested by PlaidML via an invoker. Interfacing with a frontend thus requires a Context that is linked to a Device and Invokers. A frontend implementation will need to construct instances of all these classes.

A frontend typically uses a single Context and Device; the Context can be constructed directly e.g. _ctx = plaidml.Context(), while the Device needs to be initialized with a Context and configuration settings (see _device() in the Keras frontend or _get_device_configs and PlaidMLBackend.prepare in the ONNX frontend for examples).

Functions

The Function object holds the computation graph of a network. Functions are typically constructed using compose. This requires lists of the initial input and final output variables.

The inputs are provided as pairs of variable names and placeholders. The placeholders do not yet include the input data but do tell PlaidML what format of input data to expect. At a minimum, the number of dimensions of each input must be provided; if this is all you have, inputs can be constructed with Value.from_ndims. If you know the size of one or more of the dimensions, it is better to provide the shape by constructing with Value.from_dimensions. You may also need to provide type information (see DTypes).

Weights may be included as inputs in addition to the main input data, if the weights are expected to change between runs (e.g. in training). Data that will be constant between runs should be used in constructing the output variables instead.

The outputs are provided as pairs of variable names and operation output Values (returned from an Operation via sole_output(), outputs, or output_tuple). Only the output variables returned to the user are provided here; intermediate outputs are used only in the construction of Operations, as discussed in Operations & Values.

Side effects can be also be built into a Function via the update parameter.

You must also provide a context and device (see Context & Device).

Invokers

Invokers are used to execute Functions. An Invoker is constructed from a Function and a Context, and must be provided with concrete input data to fill in the input variable placeholders in the Function and with output variables which will receive the output data produced by the Function. Different input and output variables may be used each time the Invoker is invoked.

Operations & Values

Implementing Operations for a frontend involves two broad tasks: providing an implementation of each type of operation the frontend may request, which is discussed in Individual Operations, and connecting the Operations into a computation graph, which we discuss here.

The Value class is used to store and transfer PlaidML data, and the Operation class is used to manipulate data. These are connected to form a computation graph of the desired neural network. More details are available in plaidml.tile, but for the purposes of constructing a computation graph each Operation will need to be provided input data as Values in one of the following forms:
  • Input placeholders, constructed and provided to compose as discussed in Functions.
  • Constants, often constructed via Value.from_var or Value.from_python_value, or at a lower level by manipulating a Tensor. This may use an initialization function provided by the frontend.
  • The output of already-constructed Operations. This can be accessed via op.sole_output() (if the operation has only one output), by op.outputs[name] (where name is the name of the desired output variable), or by op.output_tuple[i] (where i is the index of the desired output).

This may be handled somewhat automatically by your frontend. For example, the Keras frontend API requires a few functions to be implemented (i.e. placeholder, variable, constant, zeros, …) and then uses these to provide appropriate data when calling operation-constructing functions.

DTypes

Note that Values have an associated data type, which sometimes must be manually specified. The PlaidML datatypes are specified in DType; you will probably need to create a correspondence between these and the frontend’s data types.

Individual Operations

PlaidML operations that are common to multiple frontends can be found in the plaidml.op module (a few, such as the Python numeric type operations, instead appear in plaidml.tile). Some operations can be used directly from the common ops library, e.g. for Keras the function tanh is defined as

tanh = op.tanh

and for ONNX

@staticmethod
@opset_op('Tanh')
def tanh(value):
    return (op.tanh(value),)

Others might need a thin wrapper to translate the API, e.g. for Keras sum is defined as

def sum(x, axis=None, keepdims=False):
    return op.summation(
        x, axes=axis, keepdims=keepdims, floatx=ptile.NUMPY_DTYPE_TO_PLAIDML[floatx()])

and for ONNX

@staticmethod
@opset_op('Sum')
def sum(*args):
    return (functools.reduce(lambda x, y: x + y, args),)

(note that the + in the reduce comes from plaidml.tile as Value.__add__).

Yet other operations might not exist in the common op library, and will need to be defined in whole or in part in a frontend-specific op library; e.g. the operations switch and tile for Keras and the operations flatten and split for ONNX. If the operation you wish to implement does not yet exist, you will need to write Tile code for it (see Writing Tile Code) and wrap that code with a PlaidML Operation (see Adding Operations).