---
name: codebase-analysis
description: 'Analyze the codebase structure and count lines of code by category using cloc. Use when generating code statistics, calculating test-to-application ratios, documentation-to-code ratios, or identifying generated code percentages.'
---

# Codebase Analysis Instructions

This document provides step-by-step instructions for analyzing the ArduPilot Methodic Configurator codebase structure and counting lines of code across different categories.

## Prerequisites

- Ensure `cloc` (Count Lines of Code) tool is installed:

  ```bash
  which cloc
  ```

  If not installed, install it via your package manager (e.g., `sudo apt install cloc` on Ubuntu).

## Code Categories

The codebase is organized into the following categories:

### 1. Test Code

- **Location**: `tests/` directory
- **Purpose**: Unit tests, integration tests, and test assets
- **Command**: `cloc tests/`

### 2. Main Application Code

- **Location**: `ardupilot_methodic_configurator/` directory
- **Purpose**: Core application logic, GUI, backends, and business logic
- **Command**: `cloc ardupilot_methodic_configurator/`

### 3. Documentation Files

- **Location**: Root directory and subdirectories
- **Purpose**: User manuals, guides, architecture documentation, and README files
- **File Types**: Markdown files (*.md)
- **Commands**:

  ```bash
  # Find all markdown files
  find . -name "*.md" -type f

  # Count lines in documentation
  find . -name "*.md" -type f | xargs cloc
  ```

### 4. Configuration Files

- **Location**: Root directory and subdirectories
- **Purpose**: Project configuration, build settings, CI/CD configuration, and metadata
- **File Types**: JSON files (*.json)
- **Commands**:

  ```bash
  # Find all JSON files
  find . -name "*.json" -type f

  # Count lines in configuration files
  find . -name "*.json" -type f | xargs cloc
  ```

### 5. Generated Code

- **Location**: Within `ardupilot_methodic_configurator/` directory and `scripts/` directory
- **Purpose**: Auto-generated files that should not be manually edited
- **Files** (as documented in ARCHITECTURE.md):
  - `data_model_fc_ids.py` (generated by `update_flight_controller_ids.py`)
  - `configuration_steps_strings.py` (generated by `update_configuration_steps_translation.py`)
  - `vehicle_components.py` (generated by `update_vehicle_components_translation.py`)
  - `scripts/generate_codebase_pie_chart.py` (generated by AI assistant for codebase analysis)
- **Command**:

  ```bash
  cloc ardupilot_methodic_configurator/data_model_fc_ids.py \
       ardupilot_methodic_configurator/configuration_steps_strings.py \
       ardupilot_methodic_configurator/vehicle_components.py \
       scripts/generate_codebase_pie_chart.py
  ```

### 6. Utility Scripts

- **Locations**:
  - Root directory Python scripts
  - `scripts/` directory
- **Purpose**: Build scripts, maintenance tools, development utilities
- **Commands**:

  ```bash
  # Find all Python scripts in root directory
  find . -maxdepth 1 -name "*.py" -type f

  # Count lines in root directory scripts
  cloc ./unix2dos.py ./create_pot_file.py ./insert_missing_translations.py \
       ./mavproxy_param.py ./copy_magfit_pdef_to_template_dirs.py ./setup.py \
       ./update_flight_controller_ids.py ./test_dpi_scaling.py ./mavproxy_ftp.py \
       ./extract_missing_translations.py ./post_install.py ./param_reorder.py \
       ./param_zip.py ./find_exclusive_parameter_names.py ./get_server_pem_cert.py \
       ./update_vehicle_components_translation.py ./update_vehicle_templates.py \
       ./param_filter.py ./create_mo_files.py ./copy_param_files.py \
       ./update_configuration_steps_translation.py ./merge_pot_file.py \
       ./mavftp.py ./dos2unix.py

  # Count lines in scripts directory
  cloc scripts/
  ```

## Complete Analysis Workflow

### Step 1: Identify Generated Files

First, check ARCHITECTURE.md to identify which files are auto-generated:

```bash
grep -n "autogenerated" ARCHITECTURE.md
```

### Step 2: Count Lines by Category

1. **Test Code**:

   ```bash
   cloc tests/
   ```

2. **Main Application Code**:

   ```bash
   cloc ardupilot_methodic_configurator/
   ```

3. **Documentation Files**:

   ```bash
   find . -name "*.md" -type f | xargs cloc
   ```

4. **Configuration Files**:

   ```bash
   find . -name "*.json" -type f | xargs cloc
   ```

5. **Generated Code** (subset of main application and scripts):

   ```bash
   cloc ardupilot_methodic_configurator/data_model_fc_ids.py \
        ardupilot_methodic_configurator/configuration_steps_strings.py \
        ardupilot_methodic_configurator/vehicle_components.py \
        scripts/generate_codebase_pie_chart.py
   ```

6. **Root Directory Scripts**:

   ```bash
   find . -maxdepth 1 -name "*.py" -type f | xargs cloc
   ```

7. **Scripts Directory**:

   ```bash
   cloc scripts/
   ```

### Step 3: Calculate Summary Metrics

Extract the "code" column from each cloc output to calculate:

- **Total Test Lines**: Python code lines from tests/
- **Total Application Lines**: Python code lines from ardupilot_methodic_configurator/
- **Total Documentation Lines**: Markdown lines from *.md files
- **Total Configuration Lines**: JSON lines from *.json files
- **Generated Code Lines**: Python code lines from generated files
- **Utility Script Lines**: Python code lines from root + scripts/ directories
- **Test-to-Application Ratio**: Test lines / Application lines
- **Generated Code Percentage**: (Generated lines / Application lines) × 100
- **Documentation-to-Code Ratio**: Documentation lines / (Application lines + Test lines)

## Expected Output Format

Present results in this structure:

```text
### Tests Directory:
- **Total code lines: X** (Python only)
- Additional assets: Y lines of XML, Z lines of HTML, etc.

### Main Application (ardupilot_methodic_configurator/):
- **Total Python code lines: X**
- Additional files: Y lines of XML, Z lines of JSON, etc.

### Documentation Files (*.md):
- **Total documentation lines: X** (across Y files)
- Key files: README.md (X lines), USERMANUAL.md (Y lines), etc.

### Configuration Files (*.json):
- **Total configuration lines: X** (across Y files)
- Key files: pyproject.toml equivalent configs, CI/CD configs, etc.

### Generated Code (within the application):
- **Total generated Python code lines: X**
- **Percentage of application code: Y%**
- Files: list of generated files

### Scripts (root + scripts/ directories):
- **Root directory scripts: X Python code lines** (N files)
- **Scripts directory: Y Python code lines + Z other lines** (N files)
- **Total script lines: X**

### Key Insights:
1. **Tests**: X lines - test coverage assessment
2. **Core Application**: X lines (Python only)
3. **Documentation**: X lines (comprehensive user/developer docs)
4. **Configuration**: X lines (project setup and CI/CD)
5. **Generated Code**: X lines (~Y% of application code)
6. **Utility Scripts**: X lines for build/maintenance tasks
7. **Total Python Code**: ~X lines across all categories
8. **Test-to-Application Ratio**: X:1
9. **Documentation-to-Code Ratio**: X:1
```

## Notes

- Focus on Python code lines for the main metrics
- Generated code should not be manually edited and should be counted separately from hand-written code
- Documentation quality is assessed by documentation-to-code ratio
- Configuration files include project setup, CI/CD, and metadata files
- The project structure follows clean architecture principles
- Test coverage and generated code percentages are key quality indicators
- XML files in the application directory are likely parameter definitions
- PO files are translation files for internationalization
- The `generate_codebase_pie_chart.py` script is itself a generated file and should be excluded from utility script counts

## Maintenance

This analysis should be run:

- Before major releases
- After significant code refactoring
- When evaluating code quality metrics
- For project health assessments

Update this document if:

- New generated files are added (check ARCHITECTURE.md)
- Directory structure changes significantly
- New categories of code are introduced

## Generating Visualization

A Python script is available to automatically generate a pie chart visualization of the codebase structure:

```bash
python scripts/generate_codebase_pie_chart.py
```

This script will:

- Create a pie chart showing the distribution of code across all categories
- Save the chart as both PNG (high resolution) and SVG (scalable) formats in the `images/` directory
- Display detailed analysis with key metrics and quality indicators
- Show the chart interactively (if display is available)

**Requirements**: `matplotlib` and `numpy` packages must be installed:

```bash
pip install matplotlib numpy
```

**Output files**:

- `images/codebase_structure_pie_chart.png` - High-resolution PNG for documentation
- `images/codebase_structure_pie_chart.svg` - Scalable SVG for presentations