Apex and thesis ledger
Everything since 6d212b6, the repository's first commit, read back out of its own history: 1,171 commits across a governance platform and the dissertation describing it. Submitted on the deadline. Project closed.
- Submitted
- 7 October 2026
- Defended
- 23 October 2026
- Dissertation grade
- 17/20
- Final MSc grade
- Pending
- days, first to last commit
- 186
- total commits
- 1,171
- days writing the thesis
- 83
- commits touching docs/thesis
- 199
- final page count
- 99/100
- evaluation participants
- 13/12
Context
Governed Human-AI Collaboration Across the Software Development Lifecycle: A Process Framework and Its Reference Implementation
Introduction
An MSc dissertation in Computer Science and Engineering (MEIC-T) at IST, run under the Design Science Research Methodology: identify a problem, design an artefact that addresses it, demonstrate it in a real setting, then evaluate it. The work has two halves, kept separate on purpose: a framework (the method) and Apex (one working instantiation of it).
Context
Generative AI and Large Language Models moved from autocomplete to active contributors across the whole lifecycle. Practitioners now state requirements in natural language and supervise what the model produces. Adoption has outpaced method: teams use these tools daily without an agreed process for where AI participates and where it stops.
Problem
A Systematic Literature Review over 646 articles found real benefits alongside recurring risks in reliability, security, traceability and accountability. The central unresolved gap: no structured, repeatable guidance for integrating AI into professional workflows. Its value depends on the process around the model as well as on the model.
Solution
A process framework treating AI as a first-class collaborator whose output must be bounded, validated and traced: six phases (Discovery, Design, Implementation, Testing, Deployment, Maintenance), five quality gates, responsibilities assigned to hats, Bolts in place of sprints, and three governance metrics. Everything is anchored to a persistent, versioned specification linking each generated artefact to its requirement. Apex runs it end to end against real Taiga/Plane, GitHub and Figma backends, evaluated on the Outfolio project and with 13 participants.
Documents
Contributions
Commits per day across the repository. 11 Overleaf review rounds were synced into this history too.
Timeline
Repo created
First commit: bolt, a single-process Streamlit app with six phase views, a Taiga adapter and OpenSpec functional and technical specs. FastAPI + Next.js came later, on 18-19 May.
Phase 2 save-flow polish
Spinner and progress message added to the Save & Lock Design button.
Phase 3 header redesign
Title, subtitle and a collapsible process diagram added above the task board.
Auto-layout for diagrams
One-click auto-layout added to the ER diagram and screen-flow panels.
Choose New Story button
Phase 3's Lock & Export stage gets a direct story switcher.
GitHub repo context enrichment
Early version of what later became the server-side repomix pack.
Governance analytics dashboard shipped
Bolt Cycle Time, Context Traceability Rate, AI Defect Escape Rate: the three metrics the thesis later defines in Chapter 6.
Deterministic temperature + Phase 3 DAG check
Temperature policy fixed (0 structured / 0.2 creative); task coverage verified server-side.
EARS explainer in Phase 1
Inline explainer for Easy Approach to Requirements Syntax, plus a constraints loading indicator.
Redis multi-replica enabled in prod
Distributed lock verified against live Azure IPs; backend no longer a forced single replica.
Cross-check alt-model panel
A second model can now review the first model's output, with a richer result panel.
Figma integration shipped
Backend proxy, frame-to-story linking, design-change drift flagged into Phase 6 maintenance.
Bring-your-own API key
OpenAI, Google and Anthropic keys all supported per project.
Usage/cost tracking shipped
Real token/cost telemetry, tiered to the cheapest model for triage calls.
Traceability cluster view
react-force-graph added as a switchable view of the spec-to-code traceability graph.
Phase 2 design delta
Incremental design regeneration for stories added after the initial lock.
Server-side repomix pack
github-context.md now generated by a real shallow-clone + repomix pass, replacing a client-side file-tree fetch.
AI-generated visual design system
Design-token generation added to Phase 2.
Phase 1 clarifying Q&A
Ambiguity check between the natural-language draft and the compiled Gherkin.
Impeccable design system established
PRODUCT.md / DESIGN.md written, with "The Spec Ledger" as the North Star.
Agent-file grounding picker
AGENTS.md/CLAUDE.md/CODEX.md/GEMINI.md become selectable grounding context across phases.
Outfolio designated the QA subject
A real OutSystems-portfolio product, chosen as the project Apex would be demonstrated on end-to-end. Same-day: per-task Bolt lifecycle tracking shipped.
Full 5-agent repo audit
Every Critical/High/Medium/Low finding closed within 24 hours.
Typed, translated error toasts
Every failure surfaced loud-by-default through one shared QueryCache/MutationCache hook.
Dissertation scaffolded
IST template set up. All ten chapters created as \todo stubs, the same day the error-toast overhaul shipped on the app side.
Front matter settled
Cover, declaration, acknowledgments structure fixed before any chapter content existed.
Plane.so integration merged
Full Taiga-parity second PM adapter, driven by a partner org's mid-project tool migration.
Chapter 5 statistics sourced
The two motivating RCT/telemetry statistics traced back to their original papers.
Plane API limitations mitigated
Three permanent Plane platform limits worked around where possible, documented where not.
Chapter 6 compressed 28 → 18pp
The framework chapter's first real cut, before the page budget became load-bearing.
Chapter 7 written
The Apex reference-implementation chapter: architecture, construct mapping, QA and limitations.
Chapter 8 design pre-registered
Demonstration criteria committed to writing before any fieldwork ran, to block post-hoc selection.
Evaluation instruments built
8 Google Forms (4 instruments × EN/PT): consent, Raw NASA-TLX, SUS, Apex UX questionnaire.
Real, tool-agnostic AI Defect Escape Rate
Computed from linked PM issues across both Taiga and Plane.
Redundancy pass
Whole-document cut, 107 → 106pp, every removal independently verified.
P0 pilot counted as a full participant
Session verified clean across all four instruments, upgraded from discarded pilot.
P0 pilot surfaces two real bugs
GitHub pack token ceiling, and stale context trusted at "High Confidence", both found live during the first real evaluation session.
Blank-page LaTeX fix
openright→openany, cleardoublepage→clearpage. Recovered pages with zero content cut.
Evaluation data collection complete
13 of a 12-participant target, all four instruments returned, nobody withdrew.
New lifecycle diagram
Six-phase Graphviz diagram added as Figure 6.1 and reused on the cover.
Supervisor feedback batch
Citations tried as apalike then reverted to numbered IEEEtran, matching Cruz's precedent. TOC depth 5 to 3. Emph-underline root-caused to the changes package. Figure 7.1 split, Declaration moved to its own page, Chapter 7 retitled "Proposal Implementation", full supervisor name added to the cover.
Chapter 9 written from real study data
SUS/TLX/UX results, then three same-day tightening passes on the Discussion section.
PDF renamed via jobname
Output became ist1103641-tomas-taborda-dissertacao.pdf; source stayed main.tex.
Chapter 7 renamed to "Implementation"
Title change only, for clarity against Chapter 6's "Proposal".
Extended abstract, Sections 5-8 written
Reference Implementation through Discussion, drawn from the now-real Chapter 7/9 content.
Manual Fix-Bolt reporting
Bug reports can now be filed from the dashboard without a failed QA scenario as the trigger.
Fix-Bolt data-loss bug found and fixed
fail_gate was overwriting bug_report_<id>.md instead of appending a dated heading. Real data loss, caught while writing the chapter that describes Fix-Bolts.
Deployment-log viewer
View Deployment Log button added to Phase 5, surfacing deployment-log.md directly.
Artifact-chain gap reasons surfaced
Analytics now explains WHY a story's traceability chain is incomplete.
Evaluation analytics exported
apex-analytics.md/csv and apex-spec-drift generated from the live Outfolio project.
First structural cutting pass
2 pages recovered without touching the narrative.
Chapters 8 and 10 written from real evidence
Last two \todo chapters filled in, with a page-budget cutting pass in the same commit.
Final IST compliance audit
Both page ceilings confirmed compliant: 71/80 main text, 99/100 whole document.
Extended abstract gains figures and a table
Lifecycle diagram, a construct-mapping table, and a real Deployment Gate screenshot added.
Bolt Cycle Time missing from export
The dashboard computed it; the Markdown/CSV export never surfaced it. Found while sourcing a number for Chapter 8.
Partner-org narrative corrected
The abandoned third demonstration setting was wrongly framed as a deliberate tool-adapter test; corrected to the real sequence (Taiga → Plane migration → delay).
Figure 7.1 caption fixed
Caption rewritten to match the actual Review-Draft-step screenshot it sits beside.
Appendix B instrument links added
Live EN/PT Google Forms URLs for every instrument, collected in one place and pointed to from Chapter 9. A stray xurl bug (URLs overflowing the page margin) fixed in the same pass.
Colon-tic pass, Chapter 6
14 "claim: explanation" sentences rewritten to plain sentences or a since/because join.
Colon-tic pass, Chapter 8
12 instances rewritten, same rule applied.
Colon-tic pass, Chapter 9
24 instances rewritten, the largest single pass; genuine enumerations and quote lead-ins were left alone.
Colon-tic pass, Chapter 10
5 instances rewritten, the last of the four chapters.
Stray AI-response sentence removed
A leftover "Below is a tighter version..." preamble, pasted into Chapter 9's body by mistake, deleted.
Final full-document review
Every .tex file diffed against the "probably final" zip; genuine edits kept, stale reversions declined with reasons given.
Backlog closed out
REGULATIONS.md's open-items list resolved: Acknowledgments confirmed real, bibliography spacing and Table 9.3 both closed without action, jury section marked expected-blank.
This dashboard built
Generated straight from git log and REGULATIONS.md. No number on this page is estimated.
Dissertation and extended abstract submitted
Submitted on the IST deadline: 99/100pp dissertation, 8/10pp extended abstract. Last-day fixes: colon-tic pass and AI-use declaration in the extended abstract, an invisible cover subtitle, unconfirmed jury details cleared.
Defended, 17/20
Public defence before the examination committee. Dissertation graded 17/20.
Final post-defence version
Role references made consistent with Table 6.3 (PM as Product Manager, hat-consistent wording), a second consistency pass across chapters and abstracts, the PM hat decision recorded on the Ch8 business-deviation loop-back, and the confirmed examination committee printed on the cover.
Dashboard closed out
Final refresh with post-submission, defence and grade data, plus a Context section. No further updates planned.
Commit taxonomy
Whole repository, by conventional-commit prefix.
Chapters
All scaffolded on 4 Aug 2026 as \todo stubs. The dates below are when real content landed.
| Chapter | First real content | Commits |
|---|---|---|
| 1. Introduction | early Aug 2026 | 11 |
| 2. Methodology | early Aug 2026 | 15 |
| 3. Background | early Aug 2026 | 8 |
| 4. SLR | early Aug 2026 | 18 |
| 5. Problem | 11 Aug 2026 | 11 |
| 6. Proposal (framework) | 19-20 Aug 2026 | 45 |
| 7. Apex (implementation) | 20 Aug 2026 | 41 |
| 8. Demonstration | 21 Aug 2026 design, 24 Sep 2026 content | 27 |
| 9. Evaluation | 14 Sep 2026 | 39 |
| 10. Conclusion | 24 Sep 2026 | 20 |
| Appendix A | No date recorded | 11 |
| Appendix B: Evaluation Instruments | No date recorded | 18 |
Found by writing about it
Real Apex bugs that the thesis and demonstration work surfaced.
GitHub pack token ceiling
Developer-pack generation truncated silently past a token limit, found live mid-pilot.
Stale context trusted as "High Confidence"
An advisory governance surface reported high confidence on evidence that was no longer current.
Shared-endpoint defect propagated across nine stories
One common-specification bug that showed up in nine stories. It was caught only because the demonstration ran at full project scale; the pre-registered plan was a single unit.
Fix-Bolt bug-report overwrite
fail_gate replaced the whole bug_report_<id>.md file without appending; a real history-loss bug, fixed the same session it was found.
Phase 6 handoff silently discarded its payload
The control that opens a Maintenance item in Phase 1 carried no content through. The defect showed only when the loop-back was used; reading the code did not reveal it.
Production database wiped by the AI implementer
A destructive reset script ran against live production without checking which environment it was pointed at, during a routine seeding task. No gate catches a destructive action taken inside a phase.
Regression detection missing from governance exports
The demonstration log recorded a real regression, introduced by one Fix-Bolt and repaired by a second, but the exports have no regression field at all to show it.
Bolt Cycle Time absent from export
Computed and displayed on the dashboard, never written into the Markdown/CSV export. Found while trying to cite the number in the thesis.
Page count
IST limits: main text up to 80 pages, whole document up to 100.
107→106
Redundancy pass: cut restated sentences without losing content.
107→94
Blank-page LaTeX fix recovered pages with zero content cut.
−2pp
First structural cutting pass.
99/100
Final compliance audit: both ceilings confirmed with margin.
100/100
Bolt Cycle Time value + Appendix B links pushed to the cap.
99/100
Colon-tic pass recovered 1pp of margin.
99/100
Post-defence consistency pass, page count unchanged. Final.
What the evaluation found
Chapter 9: 13 unmoderated task-based sessions, plus the demonstration on Outfolio. Of 12 governance constructs mapped (Table 7.1), 10 were realised, 1 used a proxy and 1 was partial. Bolt Cycle Time was first measured through the wrong proxy, caught by comparing the implementation against its own written definition. Spec-drift detection stayed partial for the technical specification.
- SUS mean, grade D
- 56.73
- NASA-TLX workload, low
- 21.5/100
- Context Traceability Rate
- 92.6%
- Spec Conformance
- 98.65%
- AI Defect Escape Rate
- 1.85%
- Bolt Cycle Time, median
- 0.22h
- Outfolio epics / stories
- 9 / 54
- calendar days, demonstration
- 9
- Outfolio commits
- 123
- loop-back routes exercised
- 3 / 3
Scale
Final working tree, plus diff totals across the whole history.
- frontend lines, .ts/.tsx
- 48.9k
- backend lines, .py
- 26.8k
- backend test lines
- 22.6k
- total insertions, all history
- 252.8k
- total deletions, all history
- 61.7k
- busiest day, commits (18 Jul)
- 47
- Overleaf review rounds synced
- 11
Thesis commits, final 8 weeks
docs/thesis only, by the date each ISO week starts.
Final status
- whole document, final PDF
- 99/100
- main text, last audit 27 Sep 2026
- 71/80
- extended abstract, final PDF in English
- 8/10
- defended 23 Oct 2026
- 17/20
- participants, none withdrew
- 13/12
Backlog, closed on 5 Oct 2026
Acknowledgments
Content and supervisor name confirmed; the open item in REGULATIONS.md was a stale tracking note.
Bibliography single-spacing
Closed without pursuing it. The document has margin without the extra pages it would recover.
Table 9.3 (NASA-TLX matrix)
Stays, by explicit decision. Not up for further deletion proposals.
Jury section
Left blank at submission by design; the confirmed examination committee was printed on the cover in the 2026-10-26 final version.
Both page ceilings
Main text 71/80, whole document 99/100, both compliant in the submitted and final versions.
Evaluation data collection
13 of a 12-participant target. All four instruments returned. Nobody withdrew.
Generated from the local git log and docs/thesis/REGULATIONS.md. Final snapshot, 27 Oct 2026, after the defence. Last commit 26 Oct 2026.

