Programme Insights User Guide

1. Getting Started

Everything you need to run your first assessment, navigate the dashboard, and understand what you can do with your account.

Quick Start: Your First Assessment in 15 Minutes

You can go from login to assessment findings in under 15 minutes. Here's how.

1. Log in to Programme Insights at app.programmeinsights.com. You'll land on your dashboard.
2. Select a module from the dashboard. Each module card shows its name, the criteria framework it assesses against, and the number of criteria. Pick one that matches your current need — for example, IPA Gateway Review for a government programme approaching a gateway.
3. Upload your documents. Drag and drop PDF, Word (.docx), or Excel (.xlsx) files into the upload area. Three to five key documents is enough for a first assessment.
4. Configure assessment scope. Choose Full Assessment to run against all criteria, or select specific criteria areas to focus on. For your first run, Full Assessment gives you the broadest view.
5. Start the assessment. Click Run Assessment. You'll receive an email when it's complete — typically 15–30 minutes for a small document set.
6. Review your findings in the assessment dashboard. You'll see an overall RAG rating, criteria coverage percentage, and a prioritised list of findings with evidence citations.

Your Dashboard

When you log in, the dashboard shows you everything at a glance.

Module Selection

The top of your dashboard displays available assessment modules as cards. Each card shows:

• Module name (e.g., IPA Gateway Review, Green Book Appraisal)
• Number of criteria in the framework
• Brief description of what the module assesses
• Your most recent assessment date and RAG status for that module

Recent Assessments

Below the module cards, a table lists your recent assessments with:

• Assessment name and module
• Date run
• Overall RAG rating
• Criteria coverage percentage
• Quick actions: View Results, Generate Report, Re-run

Quick Actions

From the dashboard you can:

• New Assessment — start a fresh assessment against any module
• Upload Documents — add documents to your library without starting an assessment
• View Reports — access previously generated reports
• Manage Documents — review, tag, and organise your document library

User Roles

Programme Insights has three user roles. Your role determines what you can see and do.

2. Connecting Your Documents

Programme Insights assesses your documents against criteria frameworks. The better organised your documents, the more accurate your findings. There are two ways to connect documents: SharePoint Sync for continuous monitoring, or manual upload for ad-hoc assessments.

SharePoint Sync (Recommended)

SharePoint Sync connects Programme Insights directly to your SharePoint document libraries. Documents are automatically ingested and kept up to date — when a document changes in SharePoint, the platform detects the update and can re-assess automatically.

How to Connect SharePoint

1. Navigate to Settings > Document Sources. Click Add SharePoint Connection.
2. Authenticate. Sign in with your Microsoft 365 account. Programme Insights requests read-only access to the document libraries you select — it never modifies your SharePoint content.
3. Select your SharePoint site from the list of sites you have access to.
4. Choose document libraries. Select one or more libraries to sync. You can narrow the scope to specific folders within a library.
5. Set sync frequency. Choose how often Programme Insights checks for changes:
   • Real-time — checks every 15 minutes (recommended for active programmes)
   • Daily — syncs once per day at a time you set
   • Manual — syncs only when you trigger it
6. Map document types (optional). If your library uses metadata or naming conventions, you can map them to Programme Insights document categories. This improves assessment accuracy.

Benefits of SharePoint Sync

• Continuous monitoring — know when your compliance position changes without running manual assessments
• Automatic re-assessment — when a document is updated in SharePoint, the platform can re-run affected assessments
• No manual upload — your team doesn't need to remember to upload documents before each review
• Version tracking — Programme Insights tracks which version of each document was assessed

SharePoint Troubleshooting

Manual Upload

For ad-hoc assessments or when SharePoint isn't available, upload documents directly.

Supported Formats

Upload Limits

• Per file: 50MB maximum
• Per assessment batch: 500MB maximum
• Per upload session: 50 files at once via drag-and-drop

How to Upload

1. Go to Documents > Upload or click Upload Documents from the dashboard.
2. Drag and drop files into the upload area, or click Browse Files to select from your file system.
3. Review the upload list. Each file shows a progress bar, file size, and format validation status.
4. Add tags (optional). Tagging documents by type (e.g., Business Case, Risk Register, Benefits Realisation Plan) helps the platform locate relevant evidence more efficiently.
5. Click Upload. Files are processed and added to your document library.

Document Organisation

How Programme Insights Identifies Documents

The platform analyses document content, headings, and metadata to identify what type of document it is — business case, risk register, programme schedule, and so on. You don't need to organise documents into specific folders for the platform to work.

However, you can improve accuracy by:

• Using clear file names — Strategic_Outline_Case_v2.1.docx is better than SOC_final_FINAL_v2.docx
• Adding tags when you upload — this gives the platform an explicit signal about document purpose
• Keeping one document per file — a combined document with the OBC and FBC in a single file is harder to assess than two separate files

Managing Document Versions

When you upload a new version of a document, Programme Insights keeps the previous version in your library. You can:

• View which version was used in each assessment
• Re-run a previous assessment against the updated document to see what changed
• Archive old versions to keep your library clean

3. Running Assessments

Assessments are the core of Programme Insights. You select a criteria framework, point it at your documents, and the platform produces a structured, evidence-based assessment with traceable findings.

Selecting a Module

Each module assesses your documents against a specific criteria framework. Choose the module that matches the review, gateway, or compliance requirement you're preparing for.

Configuring Your Assessment

Once you've selected a module, configure how the assessment runs.

Assessment Scope

• Full Assessment — runs against every criterion in the framework. Best for comprehensive reviews and first-time assessments.
• Selected Criteria Areas — choose specific categories or individual criteria to assess. Useful when you know which areas need attention, or when preparing for a focused review.

Custom Criteria

If you've configured custom criteria sets (see Section 6), you can select them here instead of or in addition to the standard module criteria.

Assessment Parameters

Starting the Assessment

1. Review your configuration — the summary panel shows: module, criteria count, document count, and estimated completion time.
2. Click Run Assessment.
3. You'll see a confirmation with the assessment ID and estimated completion time. An email notification is sent when findings are ready.

Understanding Assessment Progress

While assessments run, the platform streams real-time progress updates directly to your browser using SSE (Server-Sent Events). You can watch each criterion being assessed live — for example, "Assessing criterion 12 of 62: Client-Side Budget" — or close the browser and come back later. An email notification is sent when the assessment completes.

Assessment Tiers

When configuring an assessment, you choose the assessment depth tier. This determines how thoroughly each criterion is analysed.

Status Indicators

Carry-Forward: Faster Subsequent Assessments

When you re-run an assessment against the same module, Programme Insights uses carry-forward to skip criteria that aren't affected by document changes. The platform tracks evidence lineage — which document chunks were used for each criterion — and only re-assesses criteria where the underlying documents have been added, removed, or modified. This makes subsequent assessments significantly faster than the first run.

Batch Assessments

Batch assessments let you run the same criteria framework across multiple document sets in a single operation. This is useful for:

• Consultancies reviewing multiple client programmes against the same framework
• Portfolio managers comparing compliance across programmes in a portfolio
• Trend analysis — running the same assessment against different versions of your documents over time to track improvement

How to Run a Batch Assessment

1. Go to Assessments > New Batch Assessment.
2. Select a module — all assessments in the batch use the same criteria framework.
3. Add document sets. Each document set represents one programme or submission. Name each set clearly (e.g., "Programme A — Gate 3", "Programme B — Gate 3").
4. Configure parameters (same options as a single assessment — these apply to all sets in the batch).
5. Run Batch. Each document set is assessed independently. Results are presented in a comparison view when all assessments complete.

Batch Results: Comparison View

When a batch completes, you get a comparison dashboard showing:

• Side-by-side RAG ratings per document set
• Criteria coverage comparison
• Common gaps across all document sets (issues appearing everywhere)
• Unique gaps (issues specific to one programme)

4. Understanding Your Results

Assessment results are where Programme Insights delivers its value. Every finding is traceable to evidence in your documents, every score is deterministic, and every gap comes with actionable context.

The Assessment Dashboard

When an assessment completes, the dashboard gives you an at-a-glance view of your position.

Overall DCA Rating

The assessment produces an overall Delivery Confidence Assessment (DCA) rating. This is not a simple average — it combines rules-based dimension analysis with holistic professional judgement. Criteria are grouped into dimensions, per-dimension RAG ratings are calculated, showstoppers are checked, and the final DCA reflects a considered overall position:

Criteria Coverage

Criteria coverage tells you what percentage of criteria had evidence found in your documents. A 100% coverage doesn't mean everything passed — it means the platform found relevant evidence (positive or negative) for every criterion. Low coverage indicates documents that don't address the full criteria framework.

Executive Summary

The top of the assessment dashboard shows a generated executive summary with four components:

• Verdict headline — a one-sentence summary of the overall assessment position
• Key strengths — areas where evidence is strong and criteria are well met
• Critical issues — RED-rated criteria and showstoppers that need immediate attention
• Overall recommendation — a summary judgement on readiness, with suggested next steps

Criterion-by-Criterion View

Click any criterion to see its full assessment detail.

What You See for Each Criterion

Each criterion now produces two distinct ratings:

• Coverage Rating — how well the evidence in your documents covers what the criterion requires (based on sub-question scores)
• Delivery Confidence Rating — the platform's confidence in the quality and sufficiency of that evidence for a real review

Sub-Question Breakdown

Programme Insights decomposes each criterion into specific sub-questions to produce traceable, deterministic scores. Each sub-question is scored as Yes (100%), Partial (70%), or No (0%). The coverage percentage is calculated from these scores and maps to a RAG rating: GREEN at 70% or above, AMBER at 40–69%, RED below 40%.

For the example above:

In this example, the coverage score is (100 + 100 + 0 + 70) / 4 = 67.5%, which maps to AMBER. Deterministic guardrails then apply: these rules-based overrides ensure the RAG rating is consistent with the sub-question evidence, preventing the LLM from rating a criterion GREEN when the sub-question scores indicate otherwise.

This decomposition means every score is traceable: you can see exactly which sub-questions produced the overall criterion rating and verify each one against the source evidence.

Reading Evidence Citations

Every finding in Programme Insights is backed by evidence citations. Understanding how to read and verify them is essential.

What a Citation Looks Like

Each citation includes:

• Document name — the source file
• Page or section reference — where in the document
• Text excerpt — the relevant passage (typically 1–3 sentences)
• Confidence level — how certain the platform is that this evidence is relevant to the criterion

Confidence Levels

Verifying a Citation

1. Click the citation link in the findings view. This opens the source document at the referenced page.
2. Read the cited text in context. The excerpt is highlighted. Check that the surrounding paragraphs support the finding.
3. Confirm or flag. Use the thumbs-up/thumbs-down control to confirm the citation is accurate or flag it for review. Flagged citations are excluded from report generation until resolved.

Gap Analysis

A gap is a criterion where Programme Insights found no sufficient evidence in your document set. Gaps are the most actionable output of an assessment — they tell you exactly where your documentation needs work before a review.

The Gap Register

The Gap Analysis view presents all gaps as a prioritised register:

What Makes a Gap High Priority?

Gap priority is determined by:

• Criterion weight — some criteria carry more weight in the framework (e.g., IPA "deliverability" criteria are heavily weighted)
• Review impact — gaps that are commonly flagged in reviews receive higher priority
• Related gaps — if multiple criteria in the same area are missing evidence, the area as a whole is flagged

Acting on Gaps

Each gap includes a recommendation with specific guidance:

• Which document should contain this evidence
• What type of content is needed (e.g., "quantified baseline measurements", "risk mitigation actions with owners and dates")
• Whether the gap is likely a documentation gap (evidence exists but isn't documented) or a genuine programme gap (the work hasn't been done)

5. Reports

Programme Insights generates board-ready reports with traceable evidence. Every finding in a report links back to source documents, making the output suitable for governance boards, gateway preparation packs, and SRO briefings.

Generating Reports

From any completed assessment, click Generate Report to create a formatted output. You have four report types available.

Report Types

How to Generate

1. Open the completed assessment from your dashboard or the Assessments list.
2. Click Generate Report in the top-right action bar.
3. Select report type from the dropdown.
4. Configure options (for Custom Reports): select criteria areas, evidence depth, and whether to include trend data from previous assessments.
5. Click Generate. Report generation takes 1–2 minutes. You'll see a download link when it's ready.

Report Formats

PDF Output

All reports are generated as professionally formatted PDF documents with:

• Branded cover page with assessment details, date, and module name
• Table of contents with clickable section links
• Consistent formatting — RAG indicators, evidence callouts, and structured tables throughout
• Traceable citations — every finding references the source document and page
• Print-ready — A4 layout designed for printing or sharing as-is to governance boards

Data Export

In addition to PDF reports, you can export raw assessment data for integration with other tools:

• Excel export — full criteria results in a spreadsheet, useful for importing into your own tracking systems
• CSV export — gap register data for import into project management tools
• JSON API — programmatic access for organisations integrating Programme Insights with their own dashboards or reporting tools

Sharing Reports

Download

Click the download icon on any generated report to save it locally as a PDF. Reports are also stored in your Reports library for later access.

Share via Link

Generate a shareable link for any report. Recipients don't need a Programme Insights account to view the report — but you control access:

• Anyone with link — suitable for sharing with external stakeholders (link expires after 30 days)
• Specific people — enter email addresses; recipients must verify their identity to view
• Organisation only — only users with accounts in your organisation can access

Revoking Access

Shared links can be revoked at any time from Reports > Shared Links. Once revoked, the link immediately stops working.

6. Configuring Custom Criteria

Programme Insights isn't locked to predefined frameworks. The configurable engine lets you add your own criteria sets — particularly useful for consultancies with proprietary methodologies, organisations with internal governance frameworks, or programmes with bespoke requirements.

The Configurable Engine

Every standard module (IPA, Green Book, NEC4, etc.) uses the same underlying engine. Custom criteria work identically: you define the criteria, the platform decomposes them into sub-questions, matches against your documents, and produces traceable, evidence-based findings.

How to Add a Custom Criteria Set

1. Go to Settings > Criteria Management > Create New Set.
2. Name your criteria set. Use something descriptive — e.g., "Mott MacDonald P3M3 Framework" or "Internal Programme Governance Checklist".
3. Define categories. Group related criteria into categories (e.g., "Governance", "Benefits Management", "Stakeholder Engagement"). This determines how findings are organised in reports.
4. Add criteria. For each criterion, provide:
   • Reference — a unique identifier (e.g., "GOV-01", "BEN-03")
   • Description — what the criterion requires (see best practices below)
   • Category — which category it belongs to
   • Weight (optional) — relative importance for overall scoring
   • Evidence description (optional) — what type of evidence would demonstrate compliance
5. Review sub-questions. The platform automatically generates binary decomposition sub-questions for each criterion. Review them and adjust if needed — these determine exactly how the criterion is assessed.
6. Test your criteria set. Run a test assessment against a small document set to verify the criteria produce meaningful findings. Make adjustments before publishing.
7. Publish. Move the criteria set from Draft to Active. It now appears as an option when creating assessments.

Binary Decomposition: How to Write Effective Sub-Questions

The platform generates sub-questions automatically, but reviewing and refining them produces better results. Effective sub-questions are:

• Binary — answerable as yes or no, not "somewhat" or "partially"
• Specific — reference concrete evidence (documents, data, processes)
• Independent — each sub-question stands alone; the answer to one shouldn't depend on another
• Observable — the answer can be determined from document evidence, not from interviews or tacit knowledge

Best Practices for Custom Criteria

Writing Effective Criteria

Managing Criteria Sets

• Versioning — when you update criteria, the platform creates a new version. Previous assessments remain linked to the criteria version they were run against.
• Sharing — Active criteria sets can be shared across your organisation. For consultancies, you can create criteria sets that are available to specific teams or projects.
• Importing — if you have criteria in a spreadsheet, you can import via Excel upload (Settings > Criteria Management > Import). The required columns are: Reference, Description, Category.

7. Administration

Administration functions are available to users with the Admin role. This section covers user management, module configuration, and data management.

User Management

Adding Users

1. Go to Settings > Users > Invite User.
2. Enter the user's email address.
3. Select a role: Admin, Reviewer, or Viewer (see Section 1 for role descriptions).
4. Click Send Invitation. The user receives an email with a link to set up their account.

Invitations expire after 7 days. If a user hasn't accepted, you can resend from the Users list.

Changing Roles

Go to Settings > Users, find the user, and change their role from the dropdown. The change takes effect immediately — the user doesn't need to log out and back in.

Removing Access

Click Remove next to the user in the Users list. Their access is revoked immediately. Assessment results they created are not deleted — they remain in the system linked to their name for audit trail purposes.

Module Configuration

Enabling and Disabling Modules

Not every organisation needs every module. Go to Settings > Modules to enable or disable assessment modules. Disabled modules don't appear on the dashboard or in the module selection when creating assessments.

Default Assessment Parameters

Set organisation-wide defaults for assessment configuration:

• Default confidence threshold — applied to all new assessments unless overridden
• Default evidence depth — Summary, Standard, or Detailed
• Email notifications — who receives assessment completion notifications (creator only, or all team members)

Custom Criteria Sets

Admins can manage all custom criteria sets in the organisation — including those created by other users. Go to Settings > Criteria Management to view, edit, publish, or archive criteria sets.

Data Management

Assessment Data Retention

By default, all assessment data is retained indefinitely. You can configure retention policies in Settings > Data Management:

• Keep all data (default) — assessments, documents, and reports are retained until manually deleted
• Auto-archive after N months — assessments older than the specified period are archived (still accessible but moved to cold storage)
• Auto-delete after N months — assessments and associated documents are permanently deleted after the specified period

Exporting Assessment History

Export your full assessment history as an Excel spreadsheet from Settings > Data Management > Export History. The export includes:

• All assessment metadata (date, module, user, document count)
• Overall RAG ratings and criteria coverage scores
• Gap counts per category

This is useful for governance reporting, demonstrating improvement trends to boards, or migrating data if needed.

Backup and Recovery

Programme Insights automatically backs up all data daily. In the event of data loss:

• Contact support at support@programmeinsights.com
• Provide your organisation ID (found in Settings > Organisation)
• Specify the date range of the data you need recovered

Recovery typically takes 2–4 hours during business hours.

8. Troubleshooting

Common issues and how to resolve them. For issues not covered here, contact support at support@programmeinsights.com.

Assessment Issues

Document Issues

SharePoint Issues

Report Issues

Custom Criteria Issues

Account Issues

Glossary

Key terms used throughout this guide and the Programme Insights platform.

Programme Insights | programmeinsights.com
User Guide Version 1.0 | April 2026

For support, contact support@programmeinsights.com