¿Solo quieres estudiar? Entra sin inscribirte
El curso IA generativa y pensamiento dialéctico tiene una edición de lectura abierta con sus 22 lecciones. No necesitas una cuenta para consultarlas. Solo debes registrarte y matricularte en Aula Clara si quieres presentar los cuestionarios, registrar tu progreso y obtener certificado.
1.3 · Testing AI Answers and Simulators: When Metrics Disagree
Goal: determine what an AI-generated numerical result really proves.
More output does not automatically mean better service
Suppose a model proposes giving priority to short orders. It calculates a smaller average wait, then declares the policy optimal. Optimal for which objective? A customer with a long order might now wait considerably longer. A manager may care about fairness, order completion, customer abandonment and profitability, not just the mean.
A reproducible six-order example
Six jobs arrive simultaneously at a single non-preemptive server: one takes nine minutes and five take two minutes each. If the nine-minute job is processed first, waiting times in that order are 0, 9, 11, 13, 15 and 17 minutes; the average is 65/6, roughly 10.83 minutes. If short jobs go first, waits are 0, 2, 4, 6, 8 and 10 minutes; the average is 5 minutes. The long job, however, changes from zero waiting to ten minutes.
The example illustrates a tension between mean waiting time and the treatment of particular customers. It is not empirical proof that short-job priority is the right policy in a real shop. The outcome also depends on arrivals, staffing, service-time variability and the fact that our simplified server never interrupts a job.
Six tests for a generated artifact
- Define every variable, unit and observation period.
- Identify whether the unit is a customer, order, item or service station.
- Inspect staffing, parallelism, queue rules and dependencies.
- Test zero demand, overload and missing or malformed inputs.
- Compare mean, high-percentile and worst-case waiting; state your percentile convention for small samples.
- Reproduce at least one output manually or independently in a spreadsheet.
Explain the mathematical model separately from its graphical interface. Show the calculation for six known jobs, provide expected outputs and test extreme cases. Compare mean and 90th-percentile waiting times and disclose what your simulation cannot predict about an actual coffee shop.
Assignment: hand in assumptions, a reproducible test, one newly found limitation, a revised specification and a reasoned deploy-or-reject decision. A prototype made in seconds still requires validation.
Para registrar avances, presentar los cuestionarios y recibir un certificado debes inscribirte en el curso.
Inscribirme para guardar mi progreso