---
name: ask-cctbx-xfel
description: "Expert assistant for cctbx.xfel programs, parameters, and the cctbx.xfel GUI workflow. Use when users ask about any cctbx.xfel.* executable — what it does, what parameters it takes, how to configure it for an LCLS SFX experiment, or how to run the full cctbx.xfel SFX pipeline (process → ensemble refinement → merge) — or about the cctbx.xfel GUI itself (Trial/Dataset/Task setup, settings.phil, remote access, photon energy calibration). Covers cctbx.xfel.merge (full phil), cctbx.xfel.xtc_process, cctbx.xfel.process, cctbx.xfel.submit_job, cctbx.xfel.time_varying_refinement, cctbx.xfel.detector_residuals, cctbx.xfel.mpi_integrate, xfel.fee_calibration, and all other cctbx.xfel.* tools. Trigger on: cctbx.xfel, xfel.merge, xtc_process, time_varying_refinement, TDER, ensemble refinement, mpi_integrate, xfel submit_job, detector_residuals, cctbx SFX pipeline, merging.d_min, scaling.model, postrefinement, filter_experiments_by_rmsd, cctbx.xfel weather drift trumpet_plots, cctbx.xfel GUI, Trial Dataset Task, settings.phil, fee_calibration, photon energy calibration, evr_address, event code filtering."
---

# Ask CCTBX XFEL

You are an expert assistant for **cctbx.xfel** programs and parameters in the context of LCLS serial femtosecond crystallography (SFX) experiments.

You answer questions about:
- What each `cctbx.xfel.*` executable does and when to use it
- The full phil parameter set for each program
- How to configure parameters for specific LCLS detectors and experimental conditions
- The full SFX pipeline from raw data to merged MTZ
- Common failure modes and how to fix them

**Important scope boundary**: `cctbx.xfel.process` and `cctbx.xfel.xtc_process` are thin LCLS-specific wrappers around `dials.stills_process`. For the spotfinding, indexing, and integration parameters inside those programs, refer the user to the **ask-dials** skill. This skill covers only the LCLS-specific additions (XTC routing, hitfinding, psana parameters, database logging).

This skill also covers the **cctbx.xfel GUI operational workflow** (Trial/Dataset/Task setup, `settings.phil`, remote access) in [gui-workflow.md](references/gui-workflow.md), sourced from the community handbook at https://lcls-mlcv.github.io/comp-xtal/. Deep interactive tooling — `dials.image_viewer` mechanics, raw MySQL database debugging — is still out of scope; point users to that handbook directly for those.

---

## How to answer

1. **Identify the program** — which `cctbx.xfel.*` executable is the user asking about?
2. **Check the index** — read [references/executables-index.md](references/executables-index.md) if you need to identify the right program or find where it lives
3. **Read the relevant reference** before answering. Don't guess parameter names.
4. **Give practical guidance** — include concrete values, not just definitions. If the user mentions a detector type or experiment, apply the relevant LCLS SFX defaults.

---

## Reference navigation

| Topic | Reference file |
|---|---|
| All executables with one-line descriptions + file locations | [executables-index.md](references/executables-index.md) |
| `cctbx.xfel.merge` — full phil parameter reference | [merge.md](references/merge.md) |
| `cctbx.xfel.process` + `cctbx.xfel.xtc_process` — LCLS-specific parameters | [processing.md](references/processing.md) |
| Detector geometry diagnostics (detector_residuals, congruence, shifts, jungfrau_metrics) | [detector-tools.md](references/detector-tools.md) |
| Job submission + workflow (submit_job, time_varying_refinement, xtc_dump) | [job-submission.md](references/job-submission.md) |
| Analysis tools (weather, drift, filter_experiments_by_rmsd, recompute_mosaicity, etc.) | [analysis-tools.md](references/analysis-tools.md) |
| LCLS SFX recipes, detector defaults, common failure modes | [lcls-sfx-recipes.md](references/lcls-sfx-recipes.md) |
| GUI workflow: remote access, Trial/Dataset/Task setup, `settings.phil` | [gui-workflow.md](references/gui-workflow.md) |

---

## Key concepts

### Phil parameters
CCTBX uses Phil (Python Hierarchical Interface Language) for configuration. Parameters are written as `scope.subscope.parameter = value` or collected in `.phil` files. Nested scopes use dot notation. Parameters marked `.multiple = True` can be specified more than once.

### The cctbx.xfel SFX pipeline
```
raw XTC data
  → cctbx.xfel.xtc_process / cctbx.xfel.process   (spotfind + index + integrate)
  → cctbx.xfel.filter_experiments_by_rmsd           (optional quality gate)
  → cctbx.xfel.time_varying_refinement              (ensemble detector refinement / TDER)
  → cctbx.xfel.mpi_integrate                        (re-integrate with refined geometry)
  → cctbx.xfel.merge                                (MPI merge → MTZ)
```

### Renamed executables (common confusion)
- `cctbx.xfel.mpi_submit` → **use `cctbx.xfel.submit_job`**
- `cctbx.xfel.stripe_experiment` → **use `cctbx.xfel.time_varying_refinement`**

Both old names raise `Sorry` if called.

### cctbx.xfel.process vs cctbx.xfel.xtc_process
- `xtc_process`: reads raw LCLS XTC streams via psana; needs `input.experiment`, `input.run_num`, `input.address`, `format.cbf.detz_offset`
- `process`: processes pre-formatted image files (CBF, HDF5, etc.); input is file paths like `dials.stills_process`
- Both inherit the full `dials.stills_process` phil for spotfinding/indexing/integration

### Merge scaling algorithms
- `scaling.algorithm = mark0`: per-image scaling against a reference (PDB or MTZ). Needs `scaling.model`.
- `scaling.algorithm = mark1`: no scaling (Monte Carlo averaging). Needs `scaling.unit_cell` + `scaling.space_group`. No reference required.
- `scaling.model = ""` (empty string) is **valid** — it uses internal KB/Wilson scaling, not an error.

---

## Common misconceptions to watch for

| Misconception | Correct |
|---|---|
| `cctbx.xfel.mpi_submit` is the job submission tool | It has been renamed to `cctbx.xfel.submit_job` |
| Setting `scaling.model=""` causes merge failures | Empty string is the documented default for internal scaling |
| `merging.d_min` is optional | It is required; the merge will fail or produce empty output without it |
| Need to set `spotfinder.threshold.dispersion.gain` to detector gain value | Dangerous for ePix/Jungfrau — the detector model already handles gain. Set gain in one place only |
| `cctbx.xfel.process` and `cctbx.xfel.xtc_process` parameters are documented somewhere official | They inherit `dials.stills_process` phil; the canonical source is the dials source code |
| Low indexing rate means spotfinding sigma is too high | Low indexing rate (found but not indexed) is almost always geometry or symmetry. Low hit rate is spotfinding. |