Validating and calibrating the ARIP analytical core - ON-1254

Project type: Research
Desired discipline(s): Computer science, Mathematical Sciences, Operations research, Statistics / Actuarial sciences
Company: SWICH
Project Length: 6 months to 1 year
Preferred start date: As soon as possible.
Language requirement: English
Location(s): Toronto, ON, Canada
No. of positions: 2
Desired education level: PhD
Open to applicants registered at an institution outside of Canada: Yes

About the company: 

SWICH Management Services Inc. is a Canadian management consulting and technology company focused on improving how organizations conduct research, evaluate complex information, and make high-stakes decisions. Established in 2012, SWICH combines business intelligence, analytical methods, process governance, and emerging artificial intelligence technologies to develop practical decision-support solutions.

The company’s current innovation focus is the AI-Augmented Research & Insight Platform (ARIP), a governed analytical and decision-control system designed to transform unstructured ideas, research, evidence, competing explanations, scenarios, decisions, and outcomes into structured, auditable and reproducible lifecycle records. ARIP combines deterministic analytical methods with human-governed AI assistance while preserving evidence traceability, analytical integrity, and final human decision authority.

SWICH’s broader objective is to develop adaptable decision-support technology that can be applied across multiple domains, including investment analysis, project and program evaluation, procurement, strategic planning, risk management, and resource-allocation decisions.

Describe the project.: 

SWICH Management Services Inc. is developing the AI-Augmented Research & Insight Platform (ARIP), a governed decision-support system that combines human judgment, AI-assisted reasoning and deterministic analytical methods. The objective of this research project is to validate and calibrate ARIP’s analytical core so that its outputs have an empirical and scientifically defensible foundation before the platform is expanded across additional commercial domains.

The research will evaluate whether ARIP’s core analytical methods behave as intended under controlled and real-world conditions. This includes examining the formal properties and robustness of its evidence-strength, hypothesis-support, process-reliability and structural-stability methods; determining whether analytical outputs correspond meaningfully with subsequently resolved outcomes; and assessing the reliability of human- and AI-assisted likelihood judgments.

The candidate will conduct retrospective studies using resolved hypotheses evaluated from a frozen historical information boundary, supplemented where necessary by simulation and sensitivity analysis. Each case will be reconstructed using ARIP’s governed DEFINE–MEASURE–ANALYZE lifecycle so that analytical outputs can be compared with known outcomes without introducing hindsight information. The research will also compare likelihood assessments across human reviewers and AI models to evaluate consistency, variability and their influence on final analytical results.

The expected outcomes include an independent validation of ARIP’s analytical methods, an empirical calibration dataset, identification of failure conditions and sensitivity risks, and evidence-based recommendations for analytical thresholds, reference evidence mass and prior-setting policies. These findings will support refinement of the ARIP software platform and provide a validated analytical foundation for future applications in investment analysis, project and program evaluation, procurement, risk management and other complex decision environments.

Required expertise/skills: 

The ideal candidate should have strong quantitative research skills in Bayesian statistics, probability and likelihood elicitation, forecasting, decision analysis, statistical process control, and empirical model validation. Experience with calibration analysis, sensitivity analysis, simulation, retrospective study design, uncertainty quantification, and statistical testing is important for evaluating ARIP’s analytical methods and identifying robustness or failure conditions.

The candidate should be proficient in Python and/or R for statistical analysis, simulation and reproducible research, with working knowledge of tools such as Jupyter notebooks, pandas/NumPy/SciPy or equivalent R packages, and data visualization libraries. Familiarity with SQL, Excel, Git/version control, and structured data analysis would also be valuable.

Strong analytical writing and documentation skills are required, along with the ability to translate mathematical and statistical findings into clear recommendations for software implementation.

Assets: experience with human-AI decision support, inter-rater reliability, AI/LLM evaluation, experimental design, decision science, software validation, or research involving human judgment under uncertainty.