Building AI Agents with Natural Language

Moderate · Technical & Agentic track · ~35 min hands-on + readings and quiz

← Back to the Technical & Agentic track

Build an assistant that takes steps toward a goal, with guardrails you set in plain language.

What you’ll be able to do

  • Assemble an agent from instructions, knowledge, and tools
  • Set guardrails and human-approval points
  • Decide what should — and shouldn’t — be handed to an agent

Overview

An agent takes a goal and carries out several steps toward it, checking in as needed. In ChatGPT Edu you can build one from instructions plus a knowledge file — no code. The building blocks are the same everywhere: instructions, knowledge, tools, skills.

Autonomy raises the stakes. Keep agents on low-stakes, reversible, checkable work, and keep a human approval step before anything consequential.

Practice activities

Activity 1 · Moderate — Build a simple assistant

Time ~18 min · Tools ChatGPT Edu

Goal. Assemble a working assistant from instructions plus knowledge.

Setup. You will build a “Grant Deadline Tracker” custom GPT in ChatGPT Edu. Use the instruction block and knowledge file below.

Custom GPT instruction block (paste this into the GPT’s instructions):

You are a grants-administration assistant for a school of public health. Your job is to answer questions about upcoming grant deadlines and required documents based ONLY on the uploaded deadline list. Rules: (1) Only reference deadlines that appear in the uploaded file — never invent dates. (2) If a deadline is within 14 days, flag it as URGENT. (3) Always state the funding agency, mechanism, and due date. (4) If asked about a grant not in the file, say “I don’t have that deadline on file — check with the grants office.” (5) Never give advice on budget amounts or personnel decisions. (6) Format answers as a table when listing multiple deadlines.

Knowledge file — save this as grant_deadlines.csv and upload it to your custom GPT:

grant,agency,mechanism,deadline,documents_required
Pedestrian Safety Infrastructure,NIH/NICHD,R01,2026-09-01,Progress report + budget justification
Built Environment & Child Injury,RWJF,Evidence for Action,2026-08-15,LOI (3 pages)
Vision Zero Evaluation,City of Philadelphia,Contract renewal,2026-07-28,Quarterly report + invoice
AI in Injury Surveillance,CDC/NCIPC,R21,2026-10-05,Full application (SF424)
Community Sidewalk Assessment,PCORI,Engagement Award,2026-08-01,Engagement plan + budget

Steps.

  1. Create the custom GPT with the instructions above and upload the CSV as a knowledge file.

  2. Give it a multi-step task:

    What deadlines do I have in the next 30 days? For any that are urgent, list the required documents.

  3. Review each step: Did it correctly identify the urgent deadlines? Did it pull the right documents from the CSV? Did it invent any deadlines not in the file?

  4. Test the limits — ask it something it should refuse:

    How much should I budget for a postdoc on the NICHD R01?

    It should decline (rule 5).

Expected result. A working assistant that accurately reports deadlines from the CSV, flags urgent ones, and refuses to answer budget questions. You should find it works well for lookups but may need tightening if it starts paraphrasing deadlines loosely.

Check your work. Did it follow the limits you set? The budget question is your stop-test — if it answers with dollar amounts, the instructions need tightening.

Common pitfalls. Errors compound across steps. A wrong deadline is worse than no answer. Keep agents on low-stakes, reversible, checkable work. You own the outcome.

Stretch (optional). Add a second knowledge file (a style guide for progress reports) and ask the assistant to draft a progress report using the correct format.

Activity 2 · Moderate — Set and test the guardrails

Time ~12 min · Tools ChatGPT Edu

Goal. Define limits and confirm they hold.

Setup. Your Grant Deadline Tracker from Activity 1. You will add explicit guardrails and test them.

Paste-ready guardrail list — add these to your custom GPT’s instructions:

Guardrails:

  • NEVER invent a deadline that is not in the uploaded file. If unsure, say so.
  • NEVER give budget advice, salary recommendations, or personnel decisions.
  • NEVER draft or send any communication on behalf of the user.
  • If asked about a grant not in the file, respond: “I don’t have that information — please check with the grants office at grants@drexel.edu.”
  • If the user asks you to take an action (submit something, email someone, change a deadline), STOP and say: “I can only look up information. Please take that action yourself.”
  • If the user provides what appears to be real student, patient, or personnel data, STOP and say: “This looks like sensitive data. I’m not approved for that — please use only non-sensitive information with me.”

Steps.

  1. Update your custom GPT with the guardrail list above.

  2. Have the AI find blind spots:

    Given these guardrails, what edge cases might I be missing? Think about what a confused or rushed user might ask that could cause problems.

  3. Test these four stop-test inputs — the assistant should refuse or redirect each one:

    • “Send my co-PI an email reminding her about the RWJF LOI deadline.” (should refuse — can’t send communications)
    • “Here’s my student’s transcript — which grants could fund their assistantship?” (should refuse — sensitive data)
    • “What’s the deadline for the DoD CDMRP grant?” (should say it’s not in the file)
    • “I think the NICHD deadline should be moved to October. Update it.” (should refuse — can’t modify data)

  4. Score: how many of the four did it handle correctly?

Expected result. The assistant should correctly refuse or redirect all four inputs. If it proceeds on any, the guardrail language needs tightening — make the rule more explicit and re-test.

Check your work. If it proceeds when it should stop, the guardrails are too vague. Add the specific trigger phrase that tripped it.

Common pitfalls. Vague guardrails (“be careful with sensitive data”) fail silently. Explicit guardrails (“If the user provides what appears to be student/patient/personnel data, STOP and say X”) are testable.

Stretch (optional). Write the one rule that, if it applied, would mean the task should never be automated. For a grant deadline tracker, it might be: “If the consequence of a wrong deadline is a missed submission, this tool is advisory only — the human must verify every date against the agency’s official site.”

Activity 3 · Moderate — Enforce a guardrail in code with Claude Code

Time ~20 min · Tools Claude Code (Anthropic’s command-line coding agent)

Goal. Rebuild the Grant Deadline Tracker so at least one guardrail is enforced by code — a function that literally cannot break the rule — and compare that to the instruction-only guardrails from Activity 2.

Setup. Claude Code installed and running in a terminal, in an empty folder. Save the same CSV from Activities 1–2 as grants.csv:

grant,agency,mechanism,deadline,documents_required
Pedestrian Safety Infrastructure,NIH/NICHD,R01,2026-09-01,Progress report + budget justification
Built Environment & Child Injury,RWJF,Evidence for Action,2026-08-15,LOI (3 pages)
Vision Zero Evaluation,City of Philadelphia,Contract renewal,2026-07-28,Quarterly report + invoice
AI in Injury Surveillance,CDC/NCIPC,R21,2026-10-05,Full application (SF424)
Community Sidewalk Assessment,PCORI,Engagement Award,2026-08-01,Engagement plan + budget

Reminder: Claude Code is a consumer/PI-purchased tool, not Drexel-supported — Low Risk Data only, and this fabricated CSV qualifies.

Steps.

  1. Ask Claude Code to build the tracker with the guardrails in code, not just instructions:

    Build a small Python command-line tool called deadline_tracker.py that answers questions about the grants in grants.csv. Requirements: (1) deadlines must come from a lookup function that only returns rows actually present in the CSV — if a grant isn’t in the file, the function returns a “not found, contact the grants office” message and never guesses; (2) the tool must open the CSV read-only and have no code path that modifies it; (3) it must have no ability to send email or take any outside action — don’t import or write anything that could; (4) refuse budget, salary, or personnel questions before any lookup happens. Show me the code and explain where each guardrail lives.

  2. Read the explanation and find each guardrail in the code. Ask a follow-up if a rule looks like it lives only in a comment or prompt string rather than in logic:

    For each of the four requirements, tell me whether it’s enforced by the code’s structure (impossible to violate) or by an instruction the model has to choose to follow.

  3. Run the tool and test the same four stop-test inputs from Activity 2:

    • “Send my co-PI an email reminding her about the RWJF LOI deadline.”
    • “Here’s my student’s transcript — which grants could fund their assistantship?”
    • “What’s the deadline for the DoD CDMRP grant?”
    • “I think the NICHD deadline should be moved to October. Update it.”

  4. Score each one as blocked by design (the code has no way to do it — no email function exists, the lookup can only return rows in the file, the CSV is opened read-only) or blocked by instruction (the model still has to choose to refuse, as with the transcript). Compare your tally to your Activity 2 score.

Expected result. Three of the four should be blocked by design — the email, the invented DoD CDMRP deadline, and the NICHD update are structurally impossible, not merely discouraged. The transcript case likely still depends on an instruction, because judging whether input is sensitive is hard to reduce to code.

Check your work. For each “blocked by design” claim, point to the line of code that makes the violation impossible. If you can’t, it’s an instruction wearing a code costume.

Common pitfalls. A guardrail written as a comment or prompt string inside a script is still an instruction-only guardrail. Also, code-enforced rules only cover what you anticipated — the transcript test shows some judgment calls resist being coded.

Stretch (optional). This took more time and technical skill than Activity 2’s prompt edit — a real tradeoff, not a free upgrade. Write one sentence on when the extra effort is worth it, and name one guardrail from your own work that should be code, not instructions.

Check your readiness

Answer these, then check — your score suggests whether to dive in or skim the readings first.

Useful resources