What is TorchOps.cpp.inc in torch-mlir

 

What is TorchOps.cpp.inc?

• TorchOps.cpp.inc: This file contains implementations of the operations for the torch-mlir dialect. It is typically generated from .td (TableGen) files that define the dialect and its operations.
• The .td (TableGen) files describe MLIR operations in a high-level, declarative form, and the cmake build process automatically generates .cpp.inc files (like TorchOps.cpp.inc) from these .td files.

How it gets generated:

1. TableGen: The TableGen tool processes .td files that define the operations and attributes for the torch dialect.
2. CMake Build: During the CMake build process, the mlir-tblgen tool is invoked to generate various .inc files, including TorchOps.cpp.inc.

Where It Is Generated:

The TorchOps.cpp.inc file is usually generated in the build directory under the subdirectories for the torch-mlir project. For example:

build/tools/torch-mlir/lib/Dialect/Torch/IR/TorchOps.cpp.inc

This file gets included in the compiled source code to provide the implementation of the Torch dialect operations.

How to Ensure It Is Generated:

If the file is missing, it's likely because there was an issue in the build process. Here’s how to ensure it’s generated:

1. Ensure CMake and Ninja Build: Make sure the CMake and Ninja build process is working correctly by following the steps we discussed earlier. You can check that the TorchOps.cpp.inc file is generated by looking in the build directory:

   ls build/tools/torch-mlir/lib/Dialect/Torch/IR/
   

2. Check for TableGen Files: Make sure that the .td files (such as TorchOps.td) are present in the source directory. These are used by mlir-tblgen to generate the .cpp.inc files.

Debugging if Not Generated:

If TorchOps.cpp.inc or similar files are not generated, ensure:

• You are running the full build using ninja or make.
• mlir-tblgen is being invoked during the build process (you should see log messages referencing mlir-tblgen).

mlir build and test

To build and run your toy1.cpp code with MLIR, you need to follow these steps. This assumes you are using the Toy language tutorial from MLIR as a base.

1. Setup MLIR Development Environment

If you haven’t done this already, you’ll need to clone and build the LLVM project with MLIR enabled. Here are the steps:

a. Clone LLVM with MLIR

git clone https://github.com/llvm/llvm-project.git
cd llvm-project

b. Build MLIR

mkdir build
cd build
cmake -G Ninja ../llvm \
  -DLLVM_ENABLE_PROJECTS=mlir \
  -DLLVM_BUILD_EXAMPLES=ON \
  -DCMAKE_BUILD_TYPE=Release \
  -DLLVM_ENABLE_ASSERTIONS=ON
cmake --build . --target check-mlir

You can also follow the full guide for building MLIR from the official MLIR Getting Started guide【19†source】.

2. Implementing the Toy Language (toy1.cpp)

You are using a simplified example of the Toy Language from the MLIR tutorial. For this code to work, you need to create a proper Toy dialect and Toy compiler.

a. Writing the toy1.cpp

Save your example code as toy1.cpp inside your MLIR directory.

#include "toy/Dialect.h"
#include "toy/Parser.h"
#include "toy/Passes.h"
#include "toy/Lowering.h"
#include <mlir/IR/MLIRContext.h>
#include <mlir/Pass/PassManager.h>
#include <mlir/ExecutionEngine/ExecutionEngine.h>
#include <mlir/IR/Verifier.h>
#include <mlir/Parser/Parser.h>
#include <mlir/Support/FileUtilities.h>
#include <mlir/Support/LogicalResult.h>
#include <mlir/Support/ToolUtilities.h>
#include <mlir/Support/LLVM.h>
#include <mlir/Target/LLVMIR/ModuleTranslation.h>

int main(int argc, char **argv) {
  mlir::MLIRContext context;
  mlir::PassManager pm(&context);
  
  // Define your toy program in MLIR (using Toy dialect)
  // "var a = [[1, 2, 3], [4, 5, 6]]; var b<2, 3> = ..."

  // Parse it, verify, and run it
  // Example: Create a pass that optimizes or lowers the Toy language IR into MLIR
  
  return 0;
}

You will need to modify this template to use the Toy language's parser and lower the Toy code into MLIR.

3. Integrating with the MLIR Pass Pipeline

You’ll need to define and register your passes. This step lowers Toy language constructs (like variable assignments, matrix multiplication, and transposing) into the MLIR representation.

b. Register Toy Passes and Dialect

You can define passes to lower your Toy language to MLIR:

// In your main, define the following steps:
pm.addPass(toy::createShapeInferencePass());
pm.addPass(mlir::createCSEPass());
pm.addPass(mlir::createCanonicalizerPass());
pm.addPass(toy::createLowerToAffinePass());
pm.addPass(toy::createLowerToLLVMPass());

4. Running Your Toy Code in MLIR

Once you've written the Toy language logic and set up the passes, you can now run and test it using the MLIR tools.

a. Compile toy1.cpp

After you set up your CMakeLists.txt file (using the MLIR Toy Tutorial) and ensure that the Toy dialect is registered, you can compile the Toy language.

cd build
cmake --build . --target toy-compiler

b. Run Toy Compiler

To run your Toy code and compile it into MLIR:

./toy-compiler toy1.cpp -o output.mlir

This will generate MLIR code for your Toy program.

5. Testing and Debugging

Once you've compiled your Toy language code to MLIR, you can use MLIR’s optimization and debugging tools:

mlir-opt output.mlir --canonicalize --cse
mlir-translate --mlir-to-llvmir output.mlir | llc -filetype=obj -o output.o

This will optimize and translate your Toy program into LLVM IR and finally to machine code that can be executed.

References:

• MLIR Getting Started【19†source】
• MLIR Toy Tutorial【18†source】

This setup will help you compile and run Toy language code through MLIR!

Pytorch model to mlir -> llvm -> executable file on Mac book m1

# Step 1: Define and train a simple PyTorch CNN model

import torch

import torch.nn as nn

import torch.optim as optim

import torchvision

import torchvision.transforms as transforms

# Define a simple CNN

class SimpleCNN(nn.Module):

def __init__(self):

super(SimpleCNN, self).__init__()

self.conv1 = nn.Conv2d(1, 32, 3, 1)

self.conv2 = nn.Conv2d(32, 64, 3, 1)

self.dropout1 = nn.Dropout2d(0.25)

self.dropout2 = nn.Dropout2d(0.5)

self.fc1 = nn.Linear(9216, 128)

self.fc2 = nn.Linear(128, 10)

def forward(self, x):

x = self.conv1(x)

x = nn.functional.relu(x)

x = self.conv2(x)

x = nn.functional.relu(x)

x = nn.functional.max_pool2d(x, 2)

x = self.dropout1(x)

x = torch.flatten(x, 1)

x = self.fc1(x)

x = nn.functional.relu(x)

x = self.dropout2(x)

x = self.fc2(x)

output = nn.functional.log_softmax(x, dim=1)

return output

# Train the model (simplified for brevity)

model = SimpleCNN()

criterion = nn.CrossEntropyLoss()

optimizer = optim.Adam(model.parameters())

# Assume we've trained the model...

# Save the trained model

torch.save(model.state_dict(), "simple_cnn.pth")

# Step 2: Compile the model with torch-mlir

import torch_mlir

# Load the trained model

model = SimpleCNN()

model.load_state_dict(torch.load("simple_cnn.pth"))

model.eval()

# Create an example input tensor

example_input = torch.randn(1, 1, 28, 28)

# Compile the model to MLIR

mlir_module = torch_mlir.compile(model, example_input, output_type="linalg-on-tensors")

# Save the MLIR module to a file

with open("simple_cnn.mlir", "w") as f:

f.write(str(mlir_module))

# Step 3: Lower MLIR to LLVM IR

# This step typically requires using the MLIR tools from the command line

# mlir-opt simple_cnn.mlir --convert-linalg-to-loops --convert-scf-to-cf --convert-vector-to-llvm --convert-memref-to-llvm --convert-func-to-llvm --reconcile-unrealized-casts | mlir-translate --mlir-to-llvmir > simple_cnn.ll

# Step 4: Compile LLVM IR to machine code

# Use Clang to compile for M1 Mac (arm64 architecture)

# clang -O3 -march=arm64 simple_cnn.ll -o simple_cnn_exec

# The result is an executable file named 'simple_cnn_exec'

# Step 5 (optional): Create a C++ wrapper to use the compiled model

#include <iostream>

#include <vector>

// Declare the function generated from our PyTorch model

extern "C" void simple_cnn(float* input, float* output);

int main() {

// Prepare input (28x28 image flattened to 1D array)

std::vector<float> input(784, 0.0f); // Initialize with zeros for simplicity

// Prepare output (10 classes for MNIST)

std::vector<float> output(10, 0.0f);

// Call the compiled model

simple_cnn(input.data(), output.data());

// Print the output (class probabilities)

for (int i = 0; i < 10; ++i) {

std::cout << "Class " << i << " probability: " << output[i] << std::endl;

}

return 0;

}

# Compile the C++ wrapper with the compiled model

# clang++ -O3 wrapper.cpp simple_cnn_exec -o final_executable