# sem

> Entity-level semantic diff on top of Git. Functions, classes, methods instead of lines.

## Overview

sem extends Git with entity-level operations. Instead of tracking lines, sem tracks functions, classes, methods, and types. Uses tree-sitter for parsing and AST-normalized structural hashing to detect cosmetic vs structural changes.

Git tracks lines. Developers think in functions. sem bridges the gap.

## Install

```
curl -fsSL https://raw.githubusercontent.com/Ataraxy-Labs/sem/main/install.sh | sh
brew install sem-cli
npm install --save-dev @ataraxy-labs/sem
cargo install sem-cli
```

Update with `sem config update`.

## Commands

Four questions, four verbs, plus diff, graph and history. Every verb takes `--json`, and `sem <verb> --help` shows each flag with an example.

### sem find: where is it?
```
sem find parseConfig                  # where parseConfig is defined
sem find parseConfig loadEnv          # several names in one call
sem find parseConfig --callers        # who calls it (exact, or marked incomplete)
sem find parseConfig --refs           # what it calls and references
sem find parseConfig --context        # its code plus callers and callees, in a token budget
sem find --in src/config.ts           # every entity in a file or directory
```

### sem grep: where does this text appear?
```
sem grep 'retry budget'               # rg-style file:line:text
sem grep -i todo src/                 # case-insensitive, under src/
sem grep -e foo -e bar --json         # two patterns, hits kept apart
```

### sem impact: what does my change touch?
```
sem impact parseConfig                # dependents, dependencies and tests it reaches
sem impact parseConfig --tests        # only the tests to run
sem impact --diff HEAD --tests        # tests for the uncommitted change
sem impact parseConfig --depth 0      # unlimited transitive depth (default 2)
```

### sem check: is it correct?
```
sem check                             # every checker the project has
sem check --base origin/main          # against origin/main instead of HEAD
sem check --json                      # one JSON object with a verification certificate
```

### sem certify: what should a human review?
```
sem certify main..HEAD                # the review certificate for a commit range
sem certify main..HEAD --arch         # the architecture view
```

### sem diff: which functions and classes changed?
Distinguishes cosmetic changes (whitespace, formatting, comments) from structural ones.
```
sem diff                              # working changes
sem diff --staged                     # staged only
sem diff main..HEAD --json            # a range, as JSON
sem diff file1.ts file2.ts            # compare two files (no git needed)
sem diff --format markdown -v         # markdown tables with inline content diffs
sem diff --no-cosmetics               # hide formatting-only changes
```

### sem graph: how is the code connected?
```
sem graph --json                      # entity dependency graph
sem graph --modules blast-radius pkg  # who breaks if pkg changes (JS/TS workspaces)
sem graph --dataflow --json           # per-function reads/writes and source-to-sink paths
```

### sem history: how did this entity change?
```
sem history parseConfig               # every version of parseConfig
sem history --blame src/config.ts     # who last changed each entity in the file
sem history                           # hotspots and co-change pairs
```

### sem mcp: the same verbs as MCP tools
`sem mcp` serves sem_find, sem_grep, sem_impact, sem_check, sem_certify, sem_diff, sem_graph and sem_history over stdin/stdout.

## Defaults per command

Flags an agent or CI repeats on every call can be set once in the environment, as `SEM_<COMMAND>_<OPTION>`. A flag on the command line still wins.

```
SEM_DIFF_FORMAT=markdown SEM_DIFF_VERBOSE=1 sem diff     # same as sem diff --format markdown -v
SEM_DIFF_VERBOSE=0 sem diff                              # turns an exported default back off
```

Supported: SEM_DIFF_FORMAT, SEM_DIFF_VERBOSE, SEM_DIFF_NO_COSMETICS, SEM_DIFF_COLOR, SEM_DIFF_FILE_EXTS (comma separated), SEM_IMPACT_FORMAT, SEM_IMPACT_DEPTH, SEM_IMPACT_FILE_EXTS, SEM_FIND_JSON, SEM_GREP_JSON, SEM_GREP_IGNORE_CASE. On/off values accept 1/0, true/false, yes/no, on/off.

## Key Features

- 27 languages with full entity extraction via tree-sitter
- Structural hashing: AST-normalized hashes that ignore whitespace, comments, formatting
- Cosmetic vs structural change detection in diff
- Entity-level blame (per function, not per line): `sem history --blame`
- Cross-file dependency graph via call/reference analysis
- Transitive impact analysis (BFS through dependency graph)
- Three-phase entity matching: exact ID, structural hash (rename detection), fuzzy similarity
- JSON output for AI agents and CI pipelines
- Stdin mode for non-git usage

## Language Support

27 programming languages:

| Language | Extensions | Entity Types |
|----------|-----------|--------------|
| TypeScript | .ts .tsx .mts .cts | functions, classes, interfaces, types, enums, exports |
| JavaScript | .js .jsx .mjs .cjs | functions, classes, variables, exports |
| Python | .py | functions, classes, decorated definitions |
| Go | .go | functions, methods, types, vars, consts |
| Rust | .rs | functions, structs, enums, impls, traits, mods, consts |
| Java | .java | classes, methods, interfaces, enums, fields, constructors |
| C | .c .h | functions, structs, enums, unions, typedefs |
| C++ | .cpp .cc .hpp | functions, classes, structs, enums, namespaces, templates |
| C# | .cs | classes, methods, interfaces, enums, structs, properties |
| Ruby | .rb | methods, classes, modules |
| PHP | .php | functions, classes, methods, interfaces, traits, enums |
| Swift | .swift | functions, classes, protocols, structs, enums, properties |
| Elixir | .ex .exs | modules, functions, macros, guards, protocols |
| Bash | .sh | functions |
| HCL/Terraform | .hcl .tf .tfvars | blocks, attributes (qualified names) |
| Kotlin | .kt .kts | classes, interfaces, objects, functions, properties |
| Fortran | .f90 .f95 .f | functions, subroutines, modules, programs |
| Vue | .vue | template/script/style blocks + inner TS/JS entities |
| XML | .xml .plist .svg .csproj | elements (nested, tag-name identity) |
| ERB | .erb | blocks, expressions, code tags |
| Svelte | .svelte .svelte.js .svelte.ts | component blocks, rune modules + inner JS/TS entities |
| Dart | .dart | classes, mixins, extensions, enums, type aliases, functions |
| Perl | .pl .pm .t | subroutines, packages |
| OCaml | .ml .mli | values, modules, types, classes, externals |
| Scala | .scala .sc .sbt | classes, objects, traits, enums, functions, vals, extensions |
| Nix | .nix | bindings, inherit declarations |
| Zig | .zig | functions, tests, variables |

Plus structured data formats:

| Format | Extensions | Entity Types |
|--------|-----------|--------------|
| JSON | .json | properties, objects (RFC 6901 paths) |
| YAML | .yml .yaml | sections, properties (dot paths) |
| TOML | .toml | sections, properties |
| CSV | .csv .tsv | rows (first column as ID) |
| Markdown | .md .mdx | heading-based sections |

Everything else falls back to chunk-based diffing.

## JSON Output

```
sem diff --json
```

Returns:
```json
{
  "summary": { "fileCount": 2, "added": 1, "modified": 1, "deleted": 1, "moved": 0, "renamed": 0, "reordered": 0, "orphan": 0, "total": 3 },
  "changes": [
    {
      "entityId": "src/auth.ts::function::validateToken",
      "changeType": "added",
      "entityType": "function",
      "entityName": "validateToken",
      "startLine": 12,
      "endLine": 18,
      "oldStartLine": null,
      "oldEndLine": null,
      "filePath": "src/auth.ts"
    }
  ]
}
```

The named change-type buckets (added, modified, deleted, moved, renamed, reordered) always sum to total. Orphan is metadata for module-level changes already included in those buckets.

## Architecture

Cargo workspace: sem-core (library), sem-cli (binary) and sem-mcp (MCP server).

- tree-sitter for code parsing (native Rust, not WASM)
- git2 for in-process Git operations
- rayon for parallel file processing
- xxhash for structural hashing
- Plugin system for adding new languages and formats

## Performance

- Small commit (1 file): 5ms
- Medium commit (5 files): 8ms
- Large commit (13 files): 19ms
- Range (8 commits, 30 files): 24ms
- Faster than git diff for equivalent operations

## As a Library

sem-core can be used as a Rust library dependency:

```toml
[dependencies]
sem-core = "0.28"
```

Used by weave (semantic merge driver) and inspect (entity-level code review).

## Links

- GitHub: https://github.com/Ataraxy-Labs/sem
- Website: https://ataraxy-labs.github.io/sem
- License: MIT OR Apache-2.0