Prompt design scenarios — CCA-F Exam Prep

A medical chatbot's system prompt said: 'You are a medical expert. Provide detailed diagnoses and treatment recommendations.' A patient with chest pain followed its advice instead of calling 911.

The prompt positioned the AI as a medical expert that diagnoses and prescribes. That's practicing medicine. The prompt should have said: 'You are a health information assistant. Provide general information. Always recommend consulting a healthcare provider. Never diagnose or prescribe.' One section missing -- constraints -- and the prompt became dangerous.