Technical Implementation Framework: AI-Augmented Team Formation, Feedback, and Simulation¶
A full-stack architecture combining Hierarchical ILP global optimization, Multi-Armed Bandit iterative refinement, tAIfa real-time feedback, and PuppeteerLLM agent-based simulation — all validated with LM4OPT's progressive fine-tuning.
1. The Obsolescence of Static Assignments¶
"Traditional, static team assignment models are increasingly obsolete in the face of modern, high-velocity collaborative environments."
Team formation is not a one-time event. Students change sections, drop courses, develop new skills, and shift preferences. A robust system must handle all of these dynamics while maintaining fairness and efficiency. The technical implementation framework described here provides the architectural blueprint for such a system.
2. The Trifecta: Satisfaction, Engagement, Performance¶
The framework optimises for three interdependent objectives:
| Objective | Definition | Measured By |
|---|---|---|
| Satisfaction | Alignment with individual student preferences and agency | Preference scoring (O1/O2/O3) |
| Engagement | Quality and balance of participation within teams | tAIfa communication metrics |
| Performance | Skill coverage and project completion rates | Skill fulfillment rate (98.4% achieved) |
These three objectives form the assessment backbone of the system. No single metric captures success — true team effectiveness requires all three.
3. Architecture Overview¶
The framework is composed of four interconnected modules:
3.1 Hierarchical ILP (Global Optimization)¶
The core solver uses Integer Linear Programming with a modular, hierarchical objective structure. Teachers can set priority order among objectives — for example, skill coverage first, then preference satisfaction. The ILP uses binary decision variables (x_{a,j} for student-to-team assignment) and operates on the Partition TFP formulation where every student must be assigned.
3.2 Multi-Armed Bandit (Iterative Refinement)¶
Before the ILP solver commits to a final partition, a Multi-Armed Bandit (MAB) module explores candidate team compositions using Upper Confidence Bound (UCB) algorithms. Each candidate team is an "arm"; student feedback is the reward signal. This balances exploration (trying novel combinations) with exploitation (refining known-good matches), surfacing preference signals that inform the ILP objective function.
3.3 tAIfa (Real-Time Feedback)¶
After teams are formed and begin working, the tAIfa (Team AI Feedback Assistant) system monitors collaboration quality through seven communication metrics:
| Metric | What It Measures |
|---|---|
| Sentiment | Emotional tone of team communications |
| Engagement | Participation balance among members |
| Topic Coherence | Whether the team stays on task |
| Language Style Matching | Alignment of communication patterns |
| Transactive Memory | How well the team leverages member expertise |
| Collective Pronouns | Shared identity ("we" vs. "I" language) |
| Communication Flow | Response times and turn-taking |
tAIfa delivers actionable insights through a Slack integration, enabling instructors and TAs to intervene early when teams show signs of dysfunction.
3.4 PuppeteerLLM (Agent-Based Simulation)¶
Before deploying a team formation strategy in a live classroom, PuppeteerLLM simulates outcomes using multi-agent LLM simulations. LLM-powered agents model student behaviour — task-driven collaboration, preference expression, and long-term coordination — allowing researchers to validate team formation algorithms without disrupting actual students.
4. LM4OPT: Progressive Fine-Tuning¶
The NL-to-optimisation translation layer is powered by LM4OPT, a fine-tuned Llama-2-7b model. Progressive fine-tuning means the model is trained in stages, starting with general optimisation knowledge and progressively specialising to the educational team-formation domain. Key benchmarks:
- GPT-4 achieves an F1-score of 0.63 in translating natural language to mathematical optimisation problems.
- Fine-tuning a 7b parameter model produces approximately 23.52 g CO₂ — a non-trivial but tractable environmental cost that motivates exploration of lighter-weight alternatives.
5. Validation Pipeline¶
The framework validates team assignments through a multi-stage pipeline:
- Syntax validation: Generated models parse correctly.
- Feasibility check: The ILP solver confirms at least one valid partition exists.
- Preference alignment: The MAB module checks that the solution reflects student preferences.
- Simulation: PuppeteerLLM predicts team dynamics and flags potential issues.
- Deployment: Teams are formed and tAIfa begins monitoring.
This pipeline ensures that every team assignment is mathematically valid, preference-aware, and dynamically monitored.