Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions .github/workflows/binsync-compat-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
# This workflow tests declib compatibility with binsync
# It ensures that changes to declib don't break binsync functionality

name: BinSync Compatibility Tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
binsync-tests:
runs-on: ubuntu-latest

steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
- name: Set branch name
run: echo "BRANCH_NAME=${GITHUB_HEAD_REF}" >> $GITHUB_ENV
- name: Install declib from current branch
run: |
python -m pip install --upgrade pip
pip install .
- name: Install binsync and run its core tests
run: |
# Clone binsync and try to checkout the same branch if it exists
git clone https://github.com/binsync/binsync.git /tmp/binsync
cd /tmp/binsync
git checkout $BRANCH_NAME || true
pip install pytest .[extras]
# Run binsync core tests
pytest ./tests/test_client.py ./tests/test_state.py -v
34 changes: 34 additions & 0 deletions .github/workflows/core-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Core Tests

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
build:
runs-on: ${{ matrix.os }}

strategy:
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ["3.10"]

steps:
- uses: actions/checkout@v2
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v2
with:
python-version: "${{ matrix.python-version }}"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install pytest . ./examples/change_watcher_plugin/

- name: Pytest
run: |
pytest ./tests/test_artifacts.py ./tests/test_cli.py -v
58 changes: 58 additions & 0 deletions .github/workflows/dec-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# This workflow will install Python dependencies, run tests and lint with a single version of Python
# For more information see: https://help.github.com/actions/language-and-framework-guides/using-python-with-github-actions

name: Decompiler Tests
env:
BN_SERIAL: ${{ secrets.BN_SERIAL }}
BN_LICENSE: ${{ secrets.BN_LICENSE }}
TOOLING_KEY: ${{ secrets.TOOLING_KEY }}

on:
push:
branches: [ main ]
pull_request:
branches: [ main ]

jobs:
build:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v2
- name: Set up Python 3.10
uses: actions/setup-python@v2
with:
python-version: "3.10"
- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install .[test]
- name: Download BS Artifact & Install IDA
run: |
(git clone https://github.com/binsync/bs-artifacts.git /tmp/bs-artifacts && \
cd /tmp/bs-artifacts && \
./helpers/setup_ida_ci.sh)
# taken from https://github.com/mandiant/capa/blob/master/.github/workflows/tests.yml#L107-L147
- name: Install Binary Ninja
if: ${{ env.BN_SERIAL != 0 }}
run: |
mkdir ./.github/binja
curl "https://raw.githubusercontent.com/Vector35/binaryninja-api/6812c97/scripts/download_headless.py" -o ./.github/binja/download_headless.py
python ./.github/binja/download_headless.py --serial ${{ env.BN_SERIAL }} --output .github/binja/BinaryNinja-headless.zip
unzip .github/binja/BinaryNinja-headless.zip -d .github/binja/
python .github/binja/binaryninja/scripts/install_api.py --install-on-root --silent
- name: Set up Java 21
uses: actions/setup-java@v4
with:
distribution: "oracle"
java-version: "21"
- name: Install Ghidra
uses: antoniovazquezblanco/setup-ghidra@v2.0.12
with:
version: "12.0"
auth_token: ${{ secrets.GITHUB_TOKEN }}
- name: Pytest
run: |
# these two test must be run in separate python environments, due to the way ghidra bridge works
# you also must run these tests in the exact order shown here
TEST_BINARIES_DIR=/tmp/bs-artifacts/binaries pytest tests/test_decompilers.py tests/test_client_server.py -sv
40 changes: 40 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
name: Release

on:
push:
tags:
- "v**"

jobs:

release-github:
name: Create Github Release
permissions: write-all
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v2
- name: Create Release
uses: ncipollo/release-action@v1
with:
generateReleaseNotes: true

release-pypi:
name: Release pypi package
runs-on: ubuntu-latest
steps:
- name: Checkout source
uses: actions/checkout@v2
- name: Setup Python
uses: actions/setup-python@v2
with:
python-version: '3.10'
- name: Install build
run: pip install build
- name: Build dists
run: python -m build
- name: Release to PyPI
uses: pypa/gh-action-pypi-publish@27b31702a0e7fc50959f5ad993c78deac1bdfc29
with:
user: __token__
password: ${{ secrets.PYPI_API_TOKEN }}
103 changes: 103 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Overview

DecLib is a unified decompiler API that provides an abstracted interface for working with multiple decompilers (IDA Pro, Binary Ninja, Ghidra, angr-management). It enables writing plugins and scripts that work across all supported decompilers with minimal changes.

## Development Commands

### Installation and Setup
```bash
# Install declib in development mode
pip install -e .

# Install declib plugins to decompilers (required after pip install)
declib --install

# Install to specific decompiler
declib --single-decompiler-install ida /path/to/ida
```

### Testing
```bash
# Run tests with pytest
pytest tests/

# Run specific test files
pytest tests/test_artifacts.py
pytest tests/test_decompilers.py
pytest tests/test_cli.py
pytest tests/test_remote_ghidra.py
```

### Project Management
```bash
# Build the package
python -m build

# Install test dependencies
pip install -e ".[test]"

# Install ghidra dependencies
pip install -e ".[ghidra]"
```

## Architecture

### Core Components

**DecompilerInterface** (`declib/api/decompiler_interface.py`): The main abstraction layer that provides unified access to different decompilers. Can operate in GUI mode (default) or headless mode.

**ArtifactLifter** (`declib/api/artifact_lifter.py`): Handles conversion between decompiler-specific objects and DecLib artifacts.

**Artifacts** (`declib/artifacts/`): Unified data structures representing decompiler concepts:
- `Function`, `FunctionHeader`, `FunctionArgument`
- `StackVariable`, `GlobalVariable`
- `Struct`, `StructMember`, `Enum`, `Typedef`
- `Comment`, `Patch`, `Context`, `Decompilation`

**Decompiler Implementations** (`declib/decompilers/`):
- `ida/`: IDA Pro integration
- `binja/`: Binary Ninja integration
- `ghidra/`: Ghidra integration (with bridge support)
- `angr/`: angr-management integration

### Plugin System

**Decompiler Stubs** (`declib/decompiler_stubs/`): Plugin entry points for each decompiler that bootstrap DecLib functionality.

**Plugin Installer** (`declib/plugin_installer.py`): Automatically installs DecLib plugins to detected decompiler installations.

### Key Design Patterns

**Artifact Dictionary Access**: Artifacts use a lazy-loading pattern where `.items()`, `.keys()`, `.values()` return "light" objects, but `dict[key]` returns full objects (which may trigger decompilation).

**Decompiler Discovery**: Use `DecompilerInterface.discover()` to auto-detect the current decompiler environment.

**Serialization**: All artifacts support JSON/TOML serialization via `.dumps()` and `.loads()` methods.

## Development Guidelines

### Adding New Decompiler Support
1. Create new directory in `declib/decompilers/`
2. Implement `interface.py` inheriting from `DecompilerInterface`
3. Implement `artifact_lifter.py` inheriting from `ArtifactLifter`
4. Add decompiler stub in `declib/decompiler_stubs/`
5. Update `SUPPORTED_DECOMPILERS` constant

### Working with Artifacts
- Always use `deci.functions[addr]` to get full Function objects
- Use `for addr, light_func in deci.functions.items()` for iteration
- Test serialization with both JSON and TOML formats
- Validate artifacts work across all supported decompilers

### Testing Strategy
- Unit tests for artifacts (`test_artifacts.py`)
- Integration tests for decompiler interfaces (`test_decompilers.py`)
- CLI testing (`test_cli.py`)
- Remote Ghidra functionality (`test_remote_ghidra.py`)

### Environment Variables
- `GHIDRA_HEADLESS_PATH`: Path to Ghidra headless binary for headless mode
107 changes: 93 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,105 @@
# libbs (deprecated)
# DecLib
The decompiler API that works everywhere!

> **`libbs` has been renamed to [`declib`](https://github.com/binsync/declib).**
>
> This package is now a deprecation shim. Installing it will pull in `declib`
> and emit a `DeprecationWarning` on import. No further updates will be made
> to `libbs`.

## Migrate
DecLib is an abstracted decompiler API that enables you to write plugins/scripts that work, with minimal edit,
in every decompiler supported by DecLib. DecLib was originally designed to work with [BinSync](https://binsync.net), and is the backbone
for all BinSync based plugins.
As an example, with the same script, you can [redefine the types of function variables with custom structs](./examples/struct_and_variable_use.py), all in less
than 30 lines, in any supported decompilers.

## Install
```bash
pip uninstall libbs
pip install declib
```

Replace `libbs` with `declib` in your imports:
The minimum Python version is **3.10**.

## Supported Decompilers
- IDA Pro: **>= 8.4** (if you have an older version, use `v1.26.0`)
- Binary Ninja: **>= 2.4**
- angr-management: **>= 9.0**
- Ghidra: **>= 12.0** (started in PyGhidra mode)

## Usage
DecLib exposes all decompiler API through the abstract class `DecompilerInterface`. The `DecompilerInterface`
can be used in either the default mode, which assumes a GUI, or `headless` mode. In `headless` mode, the interface will
start a new process using a specified decompiler.

You can find various examples using DecLib in the [examples](./examples) folder. Examples that are plugins show off
more of the complicated API that allows you to use an abstracted UI, artifacts, and more.

If you want a simplified command line interface (especially well-suited for LLMs), see the
[`decompiler` CLI guide](./docs/decompiler_cli.md).

### UI Mode (default)
To use the same script everywhere, use the convenience function `DecompilerInterface.discover_interface()`, which will
auto find the correct interface. Copy the below code into any supported decompiler and it should run without edit.

```python
# Before
from libbs.api import DecompilerInterface
from declib.api import DecompilerInterface

deci = DecompilerInterface.discover()
for addr in deci.functions:
function = deci.functions[addr]
if function.header.type == "void":
function.header.type = "int"
deci.functions[function.addr] = function
```

# After
Note that for Ghidra in UI mode you must first start it in PyGhidra mode. You can do this by going to your install dir
and running `./support/pyghidraRun`.

### Headless Mode
To use headless mode you must specify a decompiler to use. You can get the traditional interface using the following:

```python
from declib.api import DecompilerInterface

deci = DecompilerInterface.discover(force_decompiler="ghidra", headless=True)
```

All sources, docs, examples, and tests now live in the `declib` repository.
In the case of Ghidra, you must have the environment variable `GHIDRA_INSTALL_DIR` set to the path of the Ghidra
installation (the place the `ghidraRun` script is located).

### Artifact Access Caveats
In designing the dictionaries that contain all Artifacts in a decompiler, we had a clash between ease-of-use and speed.
When accessing some artifacts like a `Function`, we must decompile the function. Decompiling is slow. Due to this issue
we slightly changed how these dictionaries work to fast accessing.

The only way to access a **full** artifact is to use the `getitem` interface of a dictionary. In practice this
looks like the following:
```python
for func_addr, light_func in deci.functions.items():
full_function = deci.function[func_addr]
```

Notice, when using the `items` function the function is `light`, meaning it does not contain stack vars and other
info. This also means using `keys`, `values`, or `list` on an artifact dictionary will have the same affect.

### Serializing Artifacts
All artifacts are serializable to the TOML and JSON formats. Serialization is done like so:
```python
from declib.artifacts import Function
import json

my_func = Function(name="my_func", addr=0x4000, size=0x10)
json_str = my_func.dumps(fmt="json")
loaded_dict = json.loads(json_str) # now loadable through normal JSON parsing
loaded_func = Function.loads(json_str, fmt="json")
```

## Sponsors
BinSync and its associated projects would not be possible without sponsorship.
In no particular order, we'd like to thank all the organizations that have previously or are currently sponsoring
one of the many BinSync projects.

<p align="center">
<img src="https://github.com/binsync/binsync/blob/main/assets/images/sponsors/nsf.png?raw=true" alt="NSF" style="height: 100px; display: inline-block; vertical-align: middle; margin-right: 40px;">
<br>
<img src="https://github.com/binsync/binsync/blob/main/assets/images/sponsors/darpa.png?raw=true" alt="DARPA" style="height: 70px; display: inline-block; vertical-align: middle; margin-right: 40px;">
<br>
<img src="https://github.com/binsync/binsync/blob/main/assets/images/sponsors/arpah.svg?raw=true" alt="ARPA-H" style="height: 50px; display: inline-block; vertical-align: middle; margin-right: 40px;">
<br>
<img src="https://github.com/binsync/binsync/blob/main/assets/images/sponsors/reveng_ai.svg?raw=true" alt="RevEng AI" style="height: 50px; display: inline-block; vertical-align: middle;">
</p>

Loading
Loading