# flowcast Flow metrics and Monte Carlo delivery forecasting straight from your issue tracker export. flowcast answers "when will it be done?" from how your team actually delivers (cycle time, throughput, WIP and aging), with no story points. It reads Jira, Linear and GitHub Projects CSV exports, computes the flow metrics, forecasts with 10,000 Monte Carlo runs, backtests its own past forecasts, and writes a self-contained HTML report with SVG charts. ## Features - **Import**: Jira (status-transitions column or a separate changelog CSV), Linear, GitHub Projects; workflow mapping config (todo / active / waiting / done / ignored statuses); issue type and team filters; per-issue timelines that handle reopens and skipped statuses. - **Metrics**: cycle time and lead time (P50/P85/P95), weekly throughput, WIP and cumulative flow data, work item aging with at-risk flags, flow efficiency by status, arrivals vs departures, Little's Law check. - **Forecasting**: "how many items by date D" and "when will N items be done" from sampled daily throughput; configurable window; weekends and holidays excluded; scope growth (split-rate) from a range or from epic history; per-team and per-epic forecasts; backtesting. - **Output**: rich tables, ASCII charts, JSON (`--json`), HTML report (CFD, cycle-time scatterplot, aging chart, throughput, forecast histograms, backtest calibration). - **Example data**: a generator for a realistic synthetic export (2 teams, 9 months, 1,200 issues with status histories) in all three CSV flavours. ## Install ```bash uv venv .venv && uv pip install -p .venv/bin/python -e '.[dev]' source .venv/bin/activate # or call .venv/bin/flowcast directly ``` Python 3.12+, dependencies: click, rich. ## Quickstart The repo ships generated data in `examples/` (regenerate with `flowcast generate examples/`). ```bash F="examples/jira_export.csv --config examples/workflow.toml --holidays examples/holidays.txt" flowcast metrics $F flowcast aging $F flowcast forecast how-many $F --days 20 --by team flowcast forecast when $F --by team --split-history flowcast backtest $F flowcast report $F -o report.html ``` ## Usage Every command takes the export file plus these common options: | Option | Meaning | |---|---| | `--profile jira\|linear\|github` | CSV flavour (default `jira`) | | `--changelog FILE` | Jira changelog CSV (`Issue key, Changed, Field, From, To`) | | `--config FILE` | Workflow mapping TOML (see below); built-in defaults cover common status names | | `--team T`, `--type T` | Filter (repeatable, case-insensitive; `--type` matches Jira type or Linear/GitHub labels) | | `--as-of YYYY-MM-DD` | Pretend today is this date (default: newest event in the data) | | `--window N` | History window in calendar days (default 90, `0` = all data) | | `--holidays FILE`, `--holiday DATE` | Holidays to exclude from forecasting history and future days | | `--include-weekends` | Treat weekends as working days | | `--json` | Machine-readable output | Forecast commands add `--runs` (10000), `--seed` (1; same seed gives the same answer) and `--by team|epic`. ### Workflow config ```toml # Maps your tracker's statuses onto the four flow categories. # active = someone is working on it (touch time) # waiting = in progress but idle (queues, blockers); counts against flow efficiency [workflow] todo = ["To Do", "Backlog", "Todo"] active = ["In Progress", "In Review", "Testing"] waiting = ["Blocked", "Ready for Review", "Ready for QA"] done = ["Done"] ignored = ["Canceled"] [filter] # types = ["Story", "Bug"] # teams = ["Atlas"] ``` Unmapped statuses are an error that lists every offender, so nothing is silently misclassified. Issues whose final status is in `ignored` (cancelled, duplicate) are dropped. ### `flowcast metrics` ``` 1175 issues, as of 2024-09-30T17:00:00, window 2024-07-03 .. 2024-09-30 finished in window: 407 (6 skipped straight to done, excluded from cycle time) open: 110 in progress: 35 Distributions ┏━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━━┓ ┃ ┃ n ┃ mean ┃ P50 ┃ P85 ┃ P95 ┃ max ┃ ┡━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━━┩ │ cycle time (days) │ 401 │ 12.0 │ 9.0 │ 19.8 │ 29.0 │ 55.0 │ │ lead time (days) │ 407 │ 34.0 │ 25.9 │ 55.3 │ 84.7 │ 144.0 │ │ throughput (items/week) │ 13 │ 31.1 │ 28.0 │ 41.2 │ 48.0 │ 51.0 │ └─────────────────────────┴─────┴──────┴──────┴──────┴──────┴───────┘ Flow efficiency: 63% (3020 active of 4795 elapsed item-days) ┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━┓ ┃ status ┃ item-days ┃ share ┃ ┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━┩ │ In Progress │ 2433 │ 51% │ │ Ready for Review │ 650 │ 14% │ │ Ready for QA │ 529 │ 11% │ │ Blocked │ 419 │ 9% │ │ Testing │ 294 │ 6% │ │ In Review │ 293 │ 6% │ └──────────────────┴───────────┴───────┘ Little's Law: avg WIP 49.1 vs throughput x cycle time = 4.46/day x 12.0 d = 53.3 (ratio 0.92, consistent) Cycle time histogram (days) 0.0 │██████████ 30 3.2 │█████████████████████ 61 6.4 │████████████████████████████████████████ 116 9.6 │██████████████████████ 65 12.8 │██████████████████ 52 16.0 │█████ 15 19.2 │█████████ 25 22.5 │███ 8 25.7 │██ 7 28.9 │██ 7 32.1 │█ 4 35.3 │ 1 38.5 │ 1 41.7 │█ 2 44.9 │█ 2 48.1 │██ 5 Throughput per week 2024-07-01 │███████████████ 19 2024-07-08 │█████████████████████ 27 2024-07-15 │███████████████████████████████ 40 2024-07-22 │████████████████████████████████████████ 51 2024-07-29 │████████████████████████████████████ 46 2024-08-05 │██████████████████████ 28 2024-08-12 │████████████████████████████ 36 2024-08-19 │██████████████████████████ 33 2024-08-26 │██████████████████████████████ 38 2024-09-02 │█████████████ 17 2024-09-09 │████████████████ 21 2024-09-16 │██████████████████ 23 2024-09-23 │████████████████████ 25 2024-09-30 │██ 3 Arrivals vs departures (last 8 weeks) ┏━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━┓ ┃ week ┃ arrivals ┃ departures ┃ net ┃ ┡━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━┩ │ 2024-08-12 │ 14 │ 36 │ -22 │ │ 2024-08-19 │ 18 │ 33 │ -15 │ │ 2024-08-26 │ 22 │ 38 │ -16 │ │ 2024-09-02 │ 16 │ 17 │ -1 │ │ 2024-09-09 │ 30 │ 21 │ +9 │ │ 2024-09-16 │ 23 │ 23 │ +0 │ │ 2024-09-23 │ 20 │ 25 │ -5 │ │ 2024-09-30 │ 8 │ 3 │ +5 │ └────────────┴──────────┴────────────┴─────┘ ``` ### `flowcast aging` Compares the age of each in-progress item with the historical cycle time percentiles: `watch` above P50, `at-risk` above P85, `critical` above P95. ``` Reference cycle time: P50=9.0d P85=19.8d P95=29.0d ┏━━━━━━━━━┳━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━┓ ┃ item ┃ team ┃ status ┃ age (d) ┃ risk ┃ ┡━━━━━━━━━╇━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━┩ │ NIM-476 │ Nimbus │ Blocked │ 53.2 │ critical │ │ NIM-545 │ Nimbus │ Blocked │ 26.3 │ at-risk │ │ ATL-556 │ Atlas │ Testing │ 21.2 │ at-risk │ │ NIM-569 │ Nimbus │ In Progress │ 14.3 │ watch │ │ NIM-572 │ Nimbus │ Testing │ 13.3 │ watch │ │ ATL-557 │ Atlas │ Testing │ 13.0 │ watch │ │ NIM-578 │ Nimbus │ In Progress │ 11.1 │ watch │ │ NIM-510 │ Nimbus │ Ready for QA │ 11.1 │ watch │ ... ``` ### `flowcast cfd` ``` 1175 │ ........... │ ..................+++++++######### │ ...................++++++++############################ │..............+++++++########################################### │..+++++++++##################################################### │################################################################ │################################################################ │################################################################ │################################################################ │################################################################ │################################################################ │################################################################ └──────────────────────────────────────────────────────────────── 2024-07-03 2024-09-30 # done + in progress . to do ``` ### `flowcast forecast how-many` `P85 = 119+ items` means 85% of simulated runs finished at least 119 items. ``` how-many forecast from 2024-09-30 to 2024-10-28 (20 working days) ┏━━━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━┓ ┃ group ┃ items/day ┃ P50 ┃ P85 ┃ P95 ┃ ┡━━━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━┩ │ Atlas │ 3.15 │ 63+ items │ 54+ items │ 49+ items │ │ Nimbus │ 3.42 │ 68+ items │ 59+ items │ 55+ items │ │ all teams │ 6.56 │ 131+ items │ 119+ items │ 112+ items │ └───────────┴───────────┴────────────┴────────────┴────────────┘ PNN = at least this many items finished in NN% of simulated runs. Distribution for all teams (items finished): 89-93 │ 3 94-99 │█ 31 100-104 │███ 114 105-110 │██████ 265 111-115 │████████████████ 698 <- P95 116-121 │████████████████████████ 1028 <- P85 122-126 │████████████████████████████████████████ 1715 127-132 │██████████████████████████████████████ 1622 <- P50 132-137 │█████████████████████████████████████ 1576 138-142 │█████████████████████████████████ 1403 143-148 │██████████████████ 772 149-153 │███████████ 486 154-159 │████ 166 160-164 │██ 91 165-170 │ 21 171-175 │ 9 ``` ### `flowcast forecast when` `--items N` forecasts a specific number; without it the open backlog (everything not done) is used. `--split-rate 1.0-1.3` multiplies the backlog by a uniformly drawn factor in each run; `--split-history` bootstraps the factor from how much each historical epic grew relative to the scope created in its first 14 days. With `--by team` the total is the date the last team finishes. With `--by epic` each epic is forecast against the whole group's throughput, and the total assumes epics are done one after another. ``` when forecast from 2024-09-30 ┏━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┳━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┓ ┃ group ┃ backlog ┃ items/day ┃ P50 ┃ P85 ┃ P95 ┃ ┡━━━━━━━━━━━━━━━━━━╇━━━━━━━━━╇━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━┩ │ Atlas │ 55 │ 3.15 │ 2024-11-08 │ 2024-11-15 (+34d) │ 2024-11-21 │ │ │ │ │ (+29d) │ │ (+38d) │ │ Nimbus │ 55 │ 3.42 │ 2024-11-06 │ 2024-11-12 (+31d) │ 2024-11-18 │ │ │ │ │ (+27d) │ │ (+35d) │ │ all teams (last │ 110 │ - │ 2024-11-11 │ 2024-11-18 (+35d) │ 2024-11-21 │ │ to finish) │ │ │ (+30d) │ │ (+38d) │ └──────────────────┴─────────┴───────────┴──────────────────┴───────────────────┴──────────────────┘ PNN = NN% of simulated runs finished on or before this date. Scope uncertainty: backlog x historical epic growth (median 1.62, max 2.06) Distribution for all teams (last to finish) (working days needed): 21-22 │█ 29 23-24 │████ 200 24-25 │█████████████████ 873 26-27 │███████████████ 810 28-29 │███████████████████████████████████████ 2058 30-30 │████████████████████████████████████████ 2114 <- P50 32-32 │███████████████████████████████ 1616 33-34 │██████████ 531 35-36 │████████████████ 854 <- P85 37-38 │█████████ 481 <- P95 38-39 │█████ 244 40-41 │█ 77 42-43 │█ 76 44-44 │ 26 46-46 │ 7 47-48 │ 4 ``` ### `flowcast backtest` Replays "how many items in the next N working days" from weekly past origins using only data available then, and checks how often reality met each confidence level. A calibrated forecaster is met about 50% / 85% / 95% of the time. The first months of the sample dataset are a ramp-up from an empty system, which the backtest honestly shows as under-forecasting. ``` Backtest: items finished in the next 20 working days ┏━━━━━━━━━━━━┳━━━━━━━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━┳━━━━━━━━┓ ┃ origin ┃ until ┃ P95+ ┃ P85+ ┃ P50+ ┃ actual ┃ ┡━━━━━━━━━━━━╇━━━━━━━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━╇━━━━━━━━┩ │ 2024-04-01 │ 2024-04-29 │ 60 │ 67 │ 79 │ 148 │ │ 2024-04-22 │ 2024-05-20 │ 88 │ 94 │ 107 │ 123 │ │ 2024-05-13 │ 2024-06-11 │ 95 │ 101 │ 114 │ 135 │ │ 2024-06-03 │ 2024-07-02 │ 101 │ 108 │ 121 │ 113 │ │ 2024-06-24 │ 2024-07-23 │ 108 │ 115 │ 127 │ 139 │ │ 2024-07-15 │ 2024-08-12 │ 111 │ 117 │ 129 │ 166 │ │ 2024-08-05 │ 2024-09-03 │ 119 │ 126 │ 139 │ 136 │ │ 2024-08-26 │ 2024-09-24 │ 121 │ 127 │ 138 │ 98 │ └────────────┴────────────┴──────┴──────┴──────┴────────┘ P50: actual >= forecast in 62% of 8 origins (calibrated would be about 50%) P85: actual >= forecast in 88% of 8 origins (calibrated would be about 85%) P95: actual >= forecast in 88% of 8 origins (calibrated would be about 95%) Median forecast error (P50 vs actual): 20% ``` ### `flowcast report` `flowcast report $F -o report.html` writes one self-contained HTML file (inline CSS and SVG, no JavaScript, light and dark themes). Pass `--split-rate` to include scope growth in the backlog forecast. ## Architecture | Module | Role | |---|---| | `importers.py` | CSV profiles to `Issue` objects (transitions, labels, team, epic) | | `workflow.py` | Status mapping and TOML config | | `timeline.py` | Transitions to segments; start, done, cycle/lead/age; reopens and skips | | `dataset.py` | Load, validate, filter, build timelines as of a date (`Context`) | | `metrics.py` | Percentiles, throughput, CFD, aging, flow efficiency, Little's Law | | `forecast.py` | Monte Carlo simulation, confidence tables, split rate | | `analysis.py` | Per-team / per-epic forecast orchestration and JSON shaping | | `backtest.py` | Rolling-origin forecast scoring | | `calendar.py` | Working days and holidays | | `ascii.py`, `svg.py`, `report.py` | Terminal charts, SVG primitives, HTML report | | `generate.py` | Synthetic export generator | | `cli.py` | Click commands | See [docs/design.md](docs/design.md) for the algorithms with worked examples. ## Tests ```bash .venv/bin/pytest -q ``` 46 tests cover timeline construction (reopens, skipped statuses, truncation), metric formulas, CFD consistency, Monte Carlo reproducibility and convergence, backtesting, the three CSV profiles and the CLI. ## Limitations - Linear and GitHub exports only carry created/started/completed timestamps, so intermediate statuses (and therefore flow efficiency and "waiting" time) are not available for them. - Items that skip straight to done count toward throughput and lead time but have no cycle time. - Throughput sampling treats days as independent; it does not model seasonality or team size changes. Pick `--window` to reflect the current team. - The split-rate multiplier scales the whole backlog, which is a coarse model of scope growth. - Epic forecasts do not model epics competing for the same capacity.