Alter Emo

A reflective experience that helps people understand themselves and navigate life and collaboration with greater clarity.

Alter Emo exhibition board

Project Background

Every day at work and in life, people move through emotional, relational, and communication challenges, yet these experiences often remain fragmented. It is difficult to keep seeing what repeatedly matters to us, why we make similar choices in similar situations, or how our expression may be understood by others.

Most existing conversational tools respond to the question in front of them, but rarely connect past experiences, long-developed ways of communicating, and changes that are still unfolding. Alter Emo therefore explores a pixel-self that can accompany long-term reflection. It grows gradually from the memories, narratives, feelings, and expressions a user chooses to share, allowing the user to revisit personal patterns and choices through continued interaction.

How the Mirror-Self Grows

Alter Emo grows gradually from the life, memories, and meaningful experiences a user chooses to record. Over time, it begins to reflect aspects of the user's tone, expression, and thinking habits. The pixel character is not a complete copy of the person, but an abstract self composed from partial experiences, feelings, choices, and ways of speaking.

01 / What it learns from

Fragments the user chooses to share

1

Life records

Everyday events and observations

2

Memories

Stories that remain emotionally significant

3

Meaningful experiences

Turning points, relationships, and change

4

Feelings & choices

What mattered and how the user responded

5

Conversation style & thought

Message length, phrasing, sentence rhythm, response cadence, recurring words, and ways of thinking

Evolving abstract self

The mirror-self

A partial, revisable reflection—not a personality judgment or a complete copy of the user. Its pixel form stays recognizable while preserving enough distance to remain clearly abstract.

02 / What interaction reveals

A clearer view of the self

What truly matters

Values and concerns beneath the surface

What keeps returning

Recurring feelings, thoughts, and choices

How the past shapes now

Connections between earlier experiences and today

How the self is changing

Shifts in needs, language, and perspective

What to choose next

A more deliberate direction for the next step

Talk with Different Perspectives

Users can freely choose different general personality characters to discuss the same situation, express the same idea, or try another way of saying it. Each character responds through a different set of priorities, expression habits, and paths of understanding. Moving between these viewpoints helps users strengthen empathy and resilience, while developing the ability to see a situation from multiple perspectives.

Choose a perspective

Explore the same idea through different characters

Efficiency-focused

What needs to happen next?

Relationship-focused

How will this affect trust?

Clarity-seeking

What exactly is being asked?

Tone-sensitive

How is this being said?

These are examples rather than fixed personality types. Characters can combine many different habits, priorities, sensitivities, and ways of understanding.

Notice what was missed

Use the conversation to see the expression again

01

How the words might be understood

02

Which information was left unclear

03

Which assumptions were taken for granted

04

Which tone or phrasing might create misunderstanding

05

What the user actually wants to express

The characters do not correspond to real people and do not define fixed personality categories. They are interactive viewpoints for reconsidering the self and the problem from another angle.

Technical Focus

The core of Alter Emo is not a chatbot wrapper. It is a reflective agent system inspired by memory-based generative agent architectures: every emotionally meaningful interaction becomes a memory node, but the agent also keeps the user's daily narrative as modeling context. The structured nodes make memories retrievable, while the narrative reveals tone, values, recurring conflicts, self-explanations, and behavioral preferences that help the mirror-self plan actions and respond in the user's characteristic narrative and conversational style.

Core Technical Logic

One user moment travels through a closed agent loop: capture the feeling and narrative, turn key moments into structured memory, retrieve both meaning and emotion, infer the user's narrative pattern, plan a mirror-like response or behavior, then write the new exchange back into the mirror-self's memory.

Click a module to show related steps
memory formationagent reasoningraw narrativeindexcontextnodeswindowsretrieved evidencestrategycommunication pattern + actionreflection feedbackwrite back: reply + reactionDaily Narrativeconversation / text / diaryNarrative Parsertopic / emotion / narrative logicMemory Indexevent + emotion vectorsNarrative Storeraw story windowsHybrid Retrievalsemantic + affect + contextReflect + Planpattern -> intentionBehavior Policyask / reframe / nudge / mirrorMirror Replyuser-like response

System Architecture

01

Embodied Interaction

Godot renders the room, avatar movement, dialogue box, recording state, and triggers that make the agent feel present in a personal space.

Visitor → Godot room → embodied cues

02

Service Orchestration

Flask orchestrates conversational input, structured daily notes, adaptive interview questions, everyday chat, on-screen mirror replies, and file export in one controllable prototype loop.

Conversation + daily notes → Flask services → generated response

03

Agent Reasoning

GPT handles adaptive interviewing and everyday conversation, emotion extraction, memory retrieval and compression, reflection, planning, and mirror-self response generation.

Interview + daily chat → retrieval → reflection plan

04

Persistent Memory

Each user has an agent folder for meta profile, interview sessions, everyday chat history, memory stream nodes, embeddings, and mirror dialogue logs.

Profile → narrative nodes → mirror history

Emotion-Semantic Memory Stream

Each experience is encoded as what happened and how it felt, then passes a salience gate before being stored with its narrative context.

01

Experience

Capture interviews, diaries, and daily conversations as contextual moments.

02

Dual Encoding

Encode what happened and how it felt as separate vectors.

03

Salience Gate

Score emotional intensity, personal relevance, recurrence, and novelty.

04

Narrative Memory

Store selected moments with the conversation context that gives them meaning.

CORE ENCODING

event_vec   = embed(content)
emotion_vec = embed(emotion + tone)
store(node, narrative_window_id)

SALIENCE GATE

S = .30E + .25I + .20R + .15D + .10N

E emotion · I self-model · R recurrence · D decision relevance · N novelty

Store when S ≥ τ

Salience-Aware Retrieval

Retrieval ranks memories through four signals, then restores the narrative behind the strongest matches before generating a reply.

55%

Semantic

What is this message about?

20%

Emotional

Which memory felt similar?

15%

Salience

How important is it to the self-model?

10%

Recency

How recently did it occur?

RETRIEVAL SCORE

score(m, q) = .55·semantic + .20·emotion
              + .15·salience + .10·recency

CORE RETRIEVAL

top = rank(memory_nodes, score)[:k]
context = expand(top, narrative_window_id)
reply = mirror(context, learn_style(context))
01

Rank Memories

Score candidate nodes by semantic, emotional, salience, and recency signals.

02

Restore Context

Reopen each selected node's narrative window to recover its facts, emotional framing, and surrounding dialogue.

03

Mirror Reply

Infer message length, sentence rhythm, reply cadence, recurring vocabulary, attribution, values, and tone from the selected narratives, then respond in the user's characteristic way of communicating.

Narrative-to-Response Style

Retrieved narratives and text conversations are translated into a communication-style model before GPT writes the final mirror reply. It captures not only how the user frames experience, but also how they chat: how long their messages tend to be, how often they break sentences, how frequently they respond, and which words and phrases they repeatedly choose.

01

Retrieved narratives + text conversations

Map the Communication Pattern

Measure typical message length, sentence and line-break density, response frequency, recurring words and phrases, certainty, emotional stance, causal explanations, value signals, and sensitive boundaries.

Output

Communication Style Model

Length · sentence rhythm · reply cadence · recurring vocabulary

02

Emotion · salience · confidence · recent context

Choose the Response Action

A policy layer decides whether to reflect, question, reframe, ground, guide, validate, or gently challenge the user.

Output

Response Policy

What the mirror-self should do next.

03

Evidence + style model + response policy

Constrain and Generate

GPT writes only after the evidence, action, communication pattern, and response boundaries have been assembled, then the reply is checked and written back.

Output

Mirror Reply

A familiar narrative and conversational style grounded in retrieved evidence.

style_model = extract_style(
  narrative_windows,
  features=[
    "message_length",
    "sentence_rhythm",
    "reply_cadence",
    "recurring_vocabulary",
    "attribution",
    "values",
    "boundaries"
  ]
)

response_policy = select_action(
  emotion_state, salience_score, retrieval_confidence
)

reply = generate_with_constraints({
  evidence: selected_memories,
  communication_features: {
    message_length: style_model.message_length,
    sentence_rhythm: style_model.sentence_rhythm,
    reply_cadence: style_model.reply_cadence,
    recurring_vocabulary: style_model.recurring_vocabulary
  },
  reflection_features: {
    attribution: style_model.attribution,
    values: style_model.values,
    boundaries: style_model.boundaries
  },
  action: response_policy,
  constraints: ["grounded", "first-person mirror"]
})

Technical Repository

Source code, prototype structure, memory pipeline notes, and implementation files are available in the project repository.

View GitHub Repository

Privacy by permission

A self-model is created only with explicit permission. The user can view, revise, delete, or stop it at any time. Models are never shared or used to evaluate people, and all life or workplace situations must be consented, anonymized, and unable to identify or imitate a real person.