CS 222: AI Agents and Simulations

Assignment 1: Creating Generative Agents

Learning Goal

This assignment aims to provide you with hands-on experience in implementing core modules of generative agents, as discussed in class. You will focus on creating agents that store and synthesize memories from their experiences, and retrieve relevant information based on interactive contexts. The end product will be a generative agent chatbot capable of conversing with you using the memories you've instilled in it.

Part 0: OpenAI API Key

1. Create an OpenAI account using your Stanford Network ID at https://platform.openai.com/api-keys
2. You will be added to our organization account for API access.
3. Contact the TAs if you have difficulty accessing the model.
4. The class has a shared budget of $400 (approximately $8 per student).
5. Monitor your token usage at https://platform.openai.com/usage to avoid depleting the shared budget.

Part 1: Setting up the Directory

1. Clone the assignment repository: https://github.com/joonspk-research/gabm-stanford-cs222
2. Set up a Python virtual environment:

   python3 -m venv .
   source myenv/bin/activate  # On Windows, use `myenv\Scripts\activate`

3. Install required packages: pip install -r requirements.txt
4. Create a settings.py file in the simulation_engine directory with the following content:

   import sys
   import os
   from pathlib import Path
   
   OPENAI_API_KEY = "[Fill in: Your OpenAI API Key]"
   KEY_OWNER = "[Fill in: Your name]"
   DEBUG = True
   MAX_CHUNK_SIZE = 4
   LLM_VERS = "gpt-4o-mini"
   BASE_DIR = f"{Path(__file__).resolve().parent.parent}"
   POPULATIONS_DIR = f"{BASE_DIR}/agent_bank/populations"
   LLM_PROMPT_DIR = f"{BASE_DIR}/simulation_engine/prompt_template"

5. Ensure settings.py is listed in your .gitignore file to protect your API key.

Part 1.5: Repository Structure

• agent_bank: Stores agent information and memories (database-like function).
• generative_agent: Contains core logic and algorithms for agent behaviors and interactions.

You will primarily edit code in the generative_agent folder and save your agents in the agent_bank folder.

Part 2: Implement the Retrieval Function

Implement three functions in generative_agent/modules/memory_stream:

1. extract_recency: Calculate a recency score for each memory node based on last access time.
2. extract_importance: Extract the pre-calculated importance score for each memory node.
   • Write the LLM prompt for generating scores in simulation_engine/prompt_template/generative_agent/memory_stream/importance_score/*.txt
3. extract_relevance: Calculate the relevance of each memory node to a given focal point using cosine similarity of vector embeddings.

Part 3: Implement the Chat Interaction

Implement key functions for dialogue engagement in generative_agent/modules/interaction:

1. _utterance_agent_desc: Generate a description of the agent, including self-description, speech pattern, and relevant memories.
2. Create the prompt template: simulation_engine/prompt_template/generative_agent/interaction/utterance/utterance_v1.txt

Remember to test your prompts using the ChatGPT web interface for optimal results.

Part 4: Interacting with Your Agents

To test your agents' fidelity, you will develop and interact with the "Matthew Jacobs" agent, a synthetic persona built with ChatGPT.

4.1 Loading Memories

1. Initialize the agent:

   GenerativeAgent("SyntheticCS222_Base", "matthew_jacobs")

2. Load memories from memories/matthew_jacobs_memories.py:

   for m in matthew_memories:
       curr_agent.remember(m)
   curr_agent.save("SyntheticCS222", "matthew_jacobs")

4.2 Interviewing the Agent

Conduct a stateless chat session with your agent using the following question sets:

Factual Questions:

factual_questions = [
    "What business did you start after working for a larger firm?",
    "What political group did you join, and what beliefs did you strengthen through this association?",
    "How did the 2008 housing crisis impact your real estate business?",
    "How long were you married to your college girlfriend before you divorced?",
    "What role did you hold during high school that taught you leadership and responsibility?",
    "What degree did you earn from the University of Texas, and how did it shape your career?",
    "What personal challenge did you face during the COVID-19 pandemic?",
    "What health issue prompted you to focus more on your physical health?",
    "How did your relationship with your daughter change as she grew older?",
    "What was your long-time dream related to property ownership, and how did you fulfill it?"
]

Reflective Questions:

reflective_questions = [
    "How has your upbringing in rural Texas, learning to hunt and fish with your father, shaped your values and worldview?",
    "In what ways did your experience as captain of your high school football team influence your leadership style in your professional and personal life?",
    "How has your divorce and your evolving relationship with your daughter impacted your sense of responsibility and personal fulfillment?",
    "How have your libertarian beliefs influenced your approach to managing your real estate business and navigating financial challenges?",
    "How has your experience with failure, such as losing the election for local office, affected your sense of purpose and resilience?"
]

4.3 Recording Responses

1. Record the agent's responses in a CSV file:
   • First column: Question
   • Second column: Agent's response
2. Save the file as: cs222_assignment_1/report/matthew_jacobs/answers__matthew_jacobs.csv

4.4 Storing Retrieved Memories

1. For each question, store the top 10 retrieved memories in a dictionary:
   • Key: Original question
   • Value: List of 10 retrieved memories
2. Save this dictionary as a JSON file: cs222_assignment_1/report/matthew_jacobs/retrieved__matthew_jacobs.json

Example Format for Retrieved Memories:

{
    "What sparked your lifelong passion for nature?": [
        "As a child, Jasmine spent countless weekends with her grandmother, exploring the forests and learning about local plant species, sparking her lifelong passion for nature.",
        "Jasmine's experience with burnout led her to take a brief sabbatical, where she traveled to national parks across the U.S., reconnecting with nature.",
        // ... (8 more memories)
    ],
    "What was your first experience with environmental activism?": [
        "In high school, Jasmine joined the Environmental Club and organized a school-wide recycling initiative, marking her first experience in environmental activism.",
        "A pivotal moment came when Jasmine led a protest against a major corporation's pollution in her city, which garnered media attention and increased her local visibility as an activist.",
        // ... (8 more memories)
    ],
    // ... (more questions and their associated memories)
}

Part 5: Reflection

Write a couple of paragraphs describing your thoughts on the retrieval function, including:

• Whether it was retrieving the right information
• What distinguishes the memories that are considered "important" versus not

Please answer these questions in cs222_assignment_1/report/report.txt

Grading

This assignment will be graded out of 12 points. The breakdown is as follows:

Total possible points: 12 (+1 bonus point)