Apex and thesis ledger

Everything since 6d212b6, the repository's first commit, read back out of its own history: 1,171 commits across a governance platform and the dissertation describing it. Submitted on the deadline. Project closed.

Submitted
7 October 2026
Defended
23 October 2026
Dissertation grade
17/20
Final MSc grade
Pending
days, first to last commit
186
total commits
1,171
days writing the thesis
83
commits touching docs/thesis
199
final page count
99/100
evaluation participants
13/12

Context

Governed Human-AI Collaboration Across the Software Development Lifecycle: A Process Framework and Its Reference Implementation

Introduction

An MSc dissertation in Computer Science and Engineering (MEIC-T) at IST, run under the Design Science Research Methodology: identify a problem, design an artefact that addresses it, demonstrate it in a real setting, then evaluate it. The work has two halves, kept separate on purpose: a framework (the method) and Apex (one working instantiation of it).

Context

Generative AI and Large Language Models moved from autocomplete to active contributors across the whole lifecycle. Practitioners now state requirements in natural language and supervise what the model produces. Adoption has outpaced method: teams use these tools daily without an agreed process for where AI participates and where it stops.

Problem

A Systematic Literature Review over 646 articles found real benefits alongside recurring risks in reliability, security, traceability and accountability. The central unresolved gap: no structured, repeatable guidance for integrating AI into professional workflows. Its value depends on the process around the model as well as on the model.

Solution

A process framework treating AI as a first-class collaborator whose output must be bounded, validated and traced: six phases (Discovery, Design, Implementation, Testing, Deployment, Maintenance), five quality gates, responsibilities assigned to hats, Bolts in place of sprints, and three governance metrics. Everything is anchored to a persistent, versioned specification linking each generated artefact to its requirement. Apex runs it end to end against real Taiga/Plane, GitHub and Figma backends, evaluated on the Outfolio project and with 13 participants.

Documents

Contributions

Commits per day across the repository. 11 Overleaf review rounds were synced into this history too.

AprMayJunJulAugSepOctMonWedFri2026-04-19: 02026-04-20: 02026-04-21: 02026-04-22: 02026-04-23: 02026-04-24: 22026-04-25: 02026-04-26: 02026-04-27: 02026-04-28: 22026-04-29: 12026-04-30: 02026-05-01: 02026-05-02: 02026-05-03: 12026-05-04: 202026-05-05: 62026-05-06: 12026-05-07: 72026-05-08: 72026-05-09: 112026-05-10: 32026-05-11: 362026-05-12: 202026-05-13: 22026-05-14: 32026-05-15: 172026-05-16: 02026-05-17: 42026-05-18: 202026-05-19: 282026-05-20: 12026-05-21: 112026-05-22: 232026-05-23: 292026-05-24: 02026-05-25: 02026-05-26: 202026-05-27: 12026-05-28: 72026-05-29: 102026-05-30: 02026-05-31: 02026-06-01: 72026-06-02: 112026-06-03: 212026-06-04: 122026-06-05: 82026-06-06: 52026-06-07: 52026-06-08: 152026-06-09: 162026-06-10: 72026-06-11: 112026-06-12: 252026-06-13: 82026-06-14: 202026-06-15: 182026-06-16: 132026-06-17: 172026-06-18: 142026-06-19: 122026-06-20: 172026-06-21: 102026-06-22: 132026-06-23: 72026-06-24: 22026-06-25: 62026-06-26: 02026-06-27: 162026-06-28: 72026-06-29: 82026-06-30: 272026-07-01: 222026-07-02: 52026-07-03: 62026-07-04: 42026-07-05: 182026-07-06: 32026-07-07: 02026-07-08: 52026-07-09: 52026-07-10: 42026-07-11: 42026-07-12: 132026-07-13: 62026-07-14: 02026-07-15: 32026-07-16: 02026-07-17: 52026-07-18: 472026-07-19: 92026-07-20: 112026-07-21: 112026-07-22: 12026-07-23: 42026-07-24: 22026-07-25: 112026-07-26: 52026-07-27: 22026-07-28: 02026-07-29: 02026-07-30: 02026-07-31: 02026-08-01: 02026-08-02: 02026-08-03: 02026-08-04: 102026-08-05: 212026-08-06: 62026-08-07: 12026-08-08: 32026-08-09: 02026-08-10: 42026-08-11: 112026-08-12: 52026-08-13: 02026-08-14: 02026-08-15: 02026-08-16: 02026-08-17: 12026-08-18: 22026-08-19: 252026-08-20: 82026-08-21: 52026-08-22: 02026-08-23: 02026-08-24: 12026-08-25: 02026-08-26: 02026-08-27: 02026-08-28: 92026-08-29: 02026-08-30: 32026-08-31: 102026-09-01: 122026-09-02: 12026-09-03: 12026-09-04: 12026-09-05: 02026-09-06: 02026-09-07: 12026-09-08: 22026-09-09: 02026-09-10: 62026-09-11: 02026-09-12: 02026-09-13: 202026-09-14: 292026-09-15: 22026-09-16: 02026-09-17: 02026-09-18: 62026-09-19: 132026-09-20: 112026-09-21: 72026-09-22: 02026-09-23: 02026-09-24: 12026-09-25: 02026-09-26: 02026-09-27: 62026-09-28: 12026-09-29: 32026-09-30: 22026-10-01: 122026-10-02: 72026-10-03: 12026-10-04: 152026-10-05: 192026-10-06: 02026-10-07: 22026-10-08: 02026-10-09: 02026-10-10: 02026-10-11: 02026-10-12: 02026-10-13: 02026-10-14: 02026-10-15: 02026-10-16: 02026-10-17: 02026-10-18: 02026-10-19: 02026-10-20: 02026-10-21: 02026-10-22: 02026-10-23: 02026-10-24: 02026-10-25: 02026-10-26: 72026-10-27: 1

Timeline

  1. Repo created

    First commit: bolt, a single-process Streamlit app with six phase views, a Taiga adapter and OpenSpec functional and technical specs. FastAPI + Next.js came later, on 18-19 May.

    App

  2. Phase 2 save-flow polish

    Spinner and progress message added to the Save & Lock Design button.

    App

  3. Phase 3 header redesign

    Title, subtitle and a collapsible process diagram added above the task board.

    App

  4. Auto-layout for diagrams

    One-click auto-layout added to the ER diagram and screen-flow panels.

    App

  5. Choose New Story button

    Phase 3's Lock & Export stage gets a direct story switcher.

    App

  6. GitHub repo context enrichment

    Early version of what later became the server-side repomix pack.

    App

  7. Governance analytics dashboard shipped

    Bolt Cycle Time, Context Traceability Rate, AI Defect Escape Rate: the three metrics the thesis later defines in Chapter 6.

    App

  8. Deterministic temperature + Phase 3 DAG check

    Temperature policy fixed (0 structured / 0.2 creative); task coverage verified server-side.

    App

  9. EARS explainer in Phase 1

    Inline explainer for Easy Approach to Requirements Syntax, plus a constraints loading indicator.

    App

  10. Redis multi-replica enabled in prod

    Distributed lock verified against live Azure IPs; backend no longer a forced single replica.

    App

  11. Cross-check alt-model panel

    A second model can now review the first model's output, with a richer result panel.

    App

  12. Figma integration shipped

    Backend proxy, frame-to-story linking, design-change drift flagged into Phase 6 maintenance.

    App

  13. Bring-your-own API key

    OpenAI, Google and Anthropic keys all supported per project.

    App

  14. Usage/cost tracking shipped

    Real token/cost telemetry, tiered to the cheapest model for triage calls.

    App

  15. Traceability cluster view

    react-force-graph added as a switchable view of the spec-to-code traceability graph.

    App

  16. Phase 2 design delta

    Incremental design regeneration for stories added after the initial lock.

    App

  17. Server-side repomix pack

    github-context.md now generated by a real shallow-clone + repomix pass, replacing a client-side file-tree fetch.

    App

  18. AI-generated visual design system

    Design-token generation added to Phase 2.

    App

  19. Phase 1 clarifying Q&A

    Ambiguity check between the natural-language draft and the compiled Gherkin.

    App

  20. Impeccable design system established

    PRODUCT.md / DESIGN.md written, with "The Spec Ledger" as the North Star.

    App

  21. Agent-file grounding picker

    AGENTS.md/CLAUDE.md/CODEX.md/GEMINI.md become selectable grounding context across phases.

    App

  22. Outfolio designated the QA subject

    A real OutSystems-portfolio product, chosen as the project Apex would be demonstrated on end-to-end. Same-day: per-task Bolt lifecycle tracking shipped.

    App

  23. Full 5-agent repo audit

    Every Critical/High/Medium/Low finding closed within 24 hours.

    App

  24. Typed, translated error toasts

    Every failure surfaced loud-by-default through one shared QueryCache/MutationCache hook.

    App

  25. Dissertation scaffolded

    IST template set up. All ten chapters created as \todo stubs, the same day the error-toast overhaul shipped on the app side.

    Thesis

  26. Front matter settled

    Cover, declaration, acknowledgments structure fixed before any chapter content existed.

    Thesis

  27. Plane.so integration merged

    Full Taiga-parity second PM adapter, driven by a partner org's mid-project tool migration.

    App

  28. Chapter 5 statistics sourced

    The two motivating RCT/telemetry statistics traced back to their original papers.

    Thesis

  29. Plane API limitations mitigated

    Three permanent Plane platform limits worked around where possible, documented where not.

    App

  30. Chapter 6 compressed 28 → 18pp

    The framework chapter's first real cut, before the page budget became load-bearing.

    Thesis

  31. Chapter 7 written

    The Apex reference-implementation chapter: architecture, construct mapping, QA and limitations.

    Thesis

  32. Chapter 8 design pre-registered

    Demonstration criteria committed to writing before any fieldwork ran, to block post-hoc selection.

    Thesis

  33. Evaluation instruments built

    8 Google Forms (4 instruments × EN/PT): consent, Raw NASA-TLX, SUS, Apex UX questionnaire.

    Thesis

  34. Real, tool-agnostic AI Defect Escape Rate

    Computed from linked PM issues across both Taiga and Plane.

    App

  35. Redundancy pass

    Whole-document cut, 107 → 106pp, every removal independently verified.

    Thesis

  36. P0 pilot counted as a full participant

    Session verified clean across all four instruments, upgraded from discarded pilot.

    Thesis

  37. P0 pilot surfaces two real bugs

    GitHub pack token ceiling, and stale context trusted at "High Confidence", both found live during the first real evaluation session.

    Bug found in use

  38. Blank-page LaTeX fix

    openright→openany, cleardoublepage→clearpage. Recovered pages with zero content cut.

    Thesis

  39. Evaluation data collection complete

    13 of a 12-participant target, all four instruments returned, nobody withdrew.

    Thesis

  40. New lifecycle diagram

    Six-phase Graphviz diagram added as Figure 6.1 and reused on the cover.

    Thesis

  41. Supervisor feedback batch

    Citations tried as apalike then reverted to numbered IEEEtran, matching Cruz's precedent. TOC depth 5 to 3. Emph-underline root-caused to the changes package. Figure 7.1 split, Declaration moved to its own page, Chapter 7 retitled "Proposal Implementation", full supervisor name added to the cover.

    Thesis

  42. Chapter 9 written from real study data

    SUS/TLX/UX results, then three same-day tightening passes on the Discussion section.

    Thesis

  43. PDF renamed via jobname

    Output became ist1103641-tomas-taborda-dissertacao.pdf; source stayed main.tex.

    Thesis

  44. Chapter 7 renamed to "Implementation"

    Title change only, for clarity against Chapter 6's "Proposal".

    Thesis

  45. Extended abstract, Sections 5-8 written

    Reference Implementation through Discussion, drawn from the now-real Chapter 7/9 content.

    Thesis

  46. Manual Fix-Bolt reporting

    Bug reports can now be filed from the dashboard without a failed QA scenario as the trigger.

    App

  47. Fix-Bolt data-loss bug found and fixed

    fail_gate was overwriting bug_report_<id>.md instead of appending a dated heading. Real data loss, caught while writing the chapter that describes Fix-Bolts.

    Bug found in use

  48. Deployment-log viewer

    View Deployment Log button added to Phase 5, surfacing deployment-log.md directly.

    App

  49. Artifact-chain gap reasons surfaced

    Analytics now explains WHY a story's traceability chain is incomplete.

    App

  50. Evaluation analytics exported

    apex-analytics.md/csv and apex-spec-drift generated from the live Outfolio project.

    Thesis

  51. First structural cutting pass

    2 pages recovered without touching the narrative.

    Thesis

  52. Chapters 8 and 10 written from real evidence

    Last two \todo chapters filled in, with a page-budget cutting pass in the same commit.

    Thesis

  53. Final IST compliance audit

    Both page ceilings confirmed compliant: 71/80 main text, 99/100 whole document.

    Thesis

  54. Extended abstract gains figures and a table

    Lifecycle diagram, a construct-mapping table, and a real Deployment Gate screenshot added.

    Thesis

  55. Bolt Cycle Time missing from export

    The dashboard computed it; the Markdown/CSV export never surfaced it. Found while sourcing a number for Chapter 8.

    Bug found in use

  56. Partner-org narrative corrected

    The abandoned third demonstration setting was wrongly framed as a deliberate tool-adapter test; corrected to the real sequence (Taiga → Plane migration → delay).

    Thesis

  57. Figure 7.1 caption fixed

    Caption rewritten to match the actual Review-Draft-step screenshot it sits beside.

    Thesis

  58. Appendix B instrument links added

    Live EN/PT Google Forms URLs for every instrument, collected in one place and pointed to from Chapter 9. A stray xurl bug (URLs overflowing the page margin) fixed in the same pass.

    Thesis

  59. Colon-tic pass, Chapter 6

    14 "claim: explanation" sentences rewritten to plain sentences or a since/because join.

    Thesis

  60. Colon-tic pass, Chapter 8

    12 instances rewritten, same rule applied.

    Thesis

  61. Colon-tic pass, Chapter 9

    24 instances rewritten, the largest single pass; genuine enumerations and quote lead-ins were left alone.

    Thesis

  62. Colon-tic pass, Chapter 10

    5 instances rewritten, the last of the four chapters.

    Thesis

  63. Stray AI-response sentence removed

    A leftover "Below is a tighter version..." preamble, pasted into Chapter 9's body by mistake, deleted.

    Thesis

  64. Final full-document review

    Every .tex file diffed against the "probably final" zip; genuine edits kept, stale reversions declined with reasons given.

    Thesis

  65. Backlog closed out

    REGULATIONS.md's open-items list resolved: Acknowledgments confirmed real, bibliography spacing and Table 9.3 both closed without action, jury section marked expected-blank.

    Thesis

  66. This dashboard built

    Generated straight from git log and REGULATIONS.md. No number on this page is estimated.

    Thesis

  67. Dissertation and extended abstract submitted

    Submitted on the IST deadline: 99/100pp dissertation, 8/10pp extended abstract. Last-day fixes: colon-tic pass and AI-use declaration in the extended abstract, an invisible cover subtitle, unconfirmed jury details cleared.

    Thesis

  68. Defended, 17/20

    Public defence before the examination committee. Dissertation graded 17/20.

    Thesis

  69. Final post-defence version

    Role references made consistent with Table 6.3 (PM as Product Manager, hat-consistent wording), a second consistency pass across chapters and abstracts, the PM hat decision recorded on the Ch8 business-deviation loop-back, and the confirmed examination committee printed on the cover.

    Thesis

  70. Dashboard closed out

    Final refresh with post-submission, defence and grade data, plus a Context section. No further updates planned.

    Thesis

Commit taxonomy

Whole repository, by conventional-commit prefix.

feat
332
docs
315
fix
310
refactor
25
test
16
chore
10

Chapters

All scaffolded on 4 Aug 2026 as \todo stubs. The dates below are when real content landed.

ChapterFirst real contentCommits
1. Introductionearly Aug 202611
2. Methodologyearly Aug 202615
3. Backgroundearly Aug 20268
4. SLRearly Aug 202618
5. Problem11 Aug 202611
6. Proposal (framework)19-20 Aug 202645
7. Apex (implementation)20 Aug 202641
8. Demonstration21 Aug 2026 design, 24 Sep 2026 content27
9. Evaluation14 Sep 202639
10. Conclusion24 Sep 202620
Appendix ANo date recorded11
Appendix B: Evaluation InstrumentsNo date recorded18

Found by writing about it

Real Apex bugs that the thesis and demonstration work surfaced.

  1. GitHub pack token ceiling

    Developer-pack generation truncated silently past a token limit, found live mid-pilot.

  2. Stale context trusted as "High Confidence"

    An advisory governance surface reported high confidence on evidence that was no longer current.

  3. Shared-endpoint defect propagated across nine stories

    One common-specification bug that showed up in nine stories. It was caught only because the demonstration ran at full project scale; the pre-registered plan was a single unit.

  4. Fix-Bolt bug-report overwrite

    fail_gate replaced the whole bug_report_<id>.md file without appending; a real history-loss bug, fixed the same session it was found.

  5. Phase 6 handoff silently discarded its payload

    The control that opens a Maintenance item in Phase 1 carried no content through. The defect showed only when the loop-back was used; reading the code did not reveal it.

  6. Production database wiped by the AI implementer

    A destructive reset script ran against live production without checking which environment it was pointed at, during a routine seeding task. No gate catches a destructive action taken inside a phase.

  7. Regression detection missing from governance exports

    The demonstration log recorded a real regression, introduced by one Fix-Bolt and repaired by a second, but the exports have no regression field at all to show it.

  8. Bolt Cycle Time absent from export

    Computed and displayed on the dashboard, never written into the Markdown/CSV export. Found while trying to cite the number in the thesis.

Page count

IST limits: main text up to 80 pages, whole document up to 100.

  1. 107→106

    Redundancy pass: cut restated sentences without losing content.

  2. 107→94

    Blank-page LaTeX fix recovered pages with zero content cut.

  3. −2pp

    First structural cutting pass.

  4. 99/100

    Final compliance audit: both ceilings confirmed with margin.

  5. 100/100

    Bolt Cycle Time value + Appendix B links pushed to the cap.

  6. 99/100

    Colon-tic pass recovered 1pp of margin.

  7. 99/100

    Post-defence consistency pass, page count unchanged. Final.

What the evaluation found

Chapter 9: 13 unmoderated task-based sessions, plus the demonstration on Outfolio. Of 12 governance constructs mapped (Table 7.1), 10 were realised, 1 used a proxy and 1 was partial. Bolt Cycle Time was first measured through the wrong proxy, caught by comparing the implementation against its own written definition. Spec-drift detection stayed partial for the technical specification.

SUS mean, grade D
56.73
NASA-TLX workload, low
21.5/100
Context Traceability Rate
92.6%
Spec Conformance
98.65%
AI Defect Escape Rate
1.85%
Bolt Cycle Time, median
0.22h
Outfolio epics / stories
9 / 54
calendar days, demonstration
9
Outfolio commits
123
loop-back routes exercised
3 / 3

Scale

Final working tree, plus diff totals across the whole history.

frontend lines, .ts/.tsx
48.9k
backend lines, .py
26.8k
backend test lines
22.6k
total insertions, all history
252.8k
total deletions, all history
61.7k
busiest day, commits (18 Jul)
47
Overleaf review rounds synced
11

Thesis commits, final 8 weeks

docs/thesis only, by the date each ISO week starts.

7 Sep
26
14 Sep
30
21 Sep
7
28 Sep
40
5 Oct
15
12 Oct
0
19 Oct
0
26 Oct
7

Final status

whole document, final PDF
99/100
main text, last audit 27 Sep 2026
71/80
extended abstract, final PDF in English
8/10
defended 23 Oct 2026
17/20
participants, none withdrew
13/12

Backlog, closed on 5 Oct 2026

  • Acknowledgments

    Content and supervisor name confirmed; the open item in REGULATIONS.md was a stale tracking note.

  • Bibliography single-spacing

    Closed without pursuing it. The document has margin without the extra pages it would recover.

  • Table 9.3 (NASA-TLX matrix)

    Stays, by explicit decision. Not up for further deletion proposals.

  • Jury section

    Left blank at submission by design; the confirmed examination committee was printed on the cover in the 2026-10-26 final version.

  • Both page ceilings

    Main text 71/80, whole document 99/100, both compliant in the submitted and final versions.

  • Evaluation data collection

    13 of a 12-participant target. All four instruments returned. Nobody withdrew.

Generated from the local git log and docs/thesis/REGULATIONS.md. Final snapshot, 27 Oct 2026, after the defence. Last commit 26 Oct 2026.