Building a YC-Level AI Chat for My Portfolio Site

1. Introduction — Why an AI Chatbot in a Portfolio?

What is the real purpose of a portfolio site? I think it's to close the information gap between visitors and myself.

Traditional portfolios are static. Clean project write-ups, a résumé, a blog. But that's one-way information delivery. When a visitor wonders "Why did this person choose this technology?", "What's their leadership style?", or "What do they value most while actually working?", static text can't answer those questions.

An AI chatbot could be different. It can deliver information conversationally, tailored to what the visitor actually cares about.

Here's the important point though: don't just become an API wrapper. Stuffing everything into a prompt and calling the Claude API is too obvious. I wanted to build a system with the technical depth I'd seen at YC startups.

This post is the story of that journey.

2. Overall Architecture — A 9-Stage Pipeline

This is the heart of it. The path a single user request travels:

사용자 입력
  ↓
[1] Rate Limiting (DDoS 방어)
  ↓
[2] Guardrails (유해/스팸 필터링)
  ↓
[3] Intent Classification (의도 분류)
  ↓
[4] Visitor Profiling (방문자 프로파일 업데이트)
  ↓
[5] Router (어떤 경로로 갈지 결정)
  ↓
[6] Semantic Cache 확인
  ↓
[7] Hybrid Retrieval (지식 기반 검색)
  ↓
[8] LLM Re-ranking (Haiku로 정렬)
  ↓
[9] Prompt Building & Caching
  ↓
LLM 생성 (Sonnet 또는 Haiku)
  ↓
응답 전달

Three design principles guide this pipeline:

First, minimize LLM calls. LLM costs scale exponentially. At each stage, I try to handle things without LLM calls wherever possible.

Second, each stage can fail independently without taking down the whole pipeline. If the embedding service goes down, Hybrid Retrieval falls back to TF-IDF and keyword matching without the neural component. Response quality drops a little, but it doesn't completely fall apart.

Third, more complex questions take more sophisticated paths. There's no reason to run LLM Re-ranking and conversation compression on a simple greeting.

3. Knowledge Base — 42 Hand-Curated Knowledge Chunks

One of the most important decisions was not to use automatic indexing. The common approach is to crawl a website and auto-embed everything. But I deliberately hand-curated 42 chunks.

Why?

Quality. Automatic indexing is noisy. Logos, navigation, ads, footer text — all of that gets mixed into the embeddings, which degrades semantic search performance.

Consistency. My portfolio is meant to convey "the story of a person named Seungwon Jeong." Automatically generated chunks lose the tone and context of that story.

Each chunk has this structure:

{
  id: "project-daval-01",
  keywords: ["DaVal", "AI", "금융", "founder"],
  content: "마크다운으로 작성된 실제 콘텐츠..."
}

Topics covered by the 42 chunks:

• Personal info (name, title, location)
• Philosophy (technical outlook, leadership style, career values)
• Key projects (DaVal, SMUSIGNAL, KMP, etc.) — each broken into multiple chunks
• Tech stack (backend, frontend, infrastructure)
• Personality and values (work style, collaboration approach, time management)
• Hobbies and personal life (reading, exercise, music)
• Frequently asked questions (hiring, collaboration proposals, technical consulting)

The real advantage of doing this by hand shows up later.

4. Hybrid Retrieval — TF-IDF + Keyword + Neural Embedding, Three Ways

What if I'd used pure vector search? Korean is especially weak with vector search on short queries. Questions like "tech stack?" or "what's your philosophy?" — five to ten characters — are hard to distinguish through embeddings alone.

So I combined three approaches.

TF-IDF (natural.js library)

Traditional statistics-based search. It scores relevance by computing word frequency. Applying natural.js's Japanese tokenizer to Korean works surprisingly well.

const TfIdf = require('natural').TfIdf;
const tfidf = new TfIdf();
 
chunks.forEach(chunk => {
  tfidf.addDocument(chunk.content);
});
 
const results = tfidf.tfidfs(userQuery).map((score, idx) => ({
  chunkId: chunks[idx].id,
  score
}));

Keyword Matching

Directly compares each chunk's keywords array against the user's question. If a "DaVal" question comes in, chunks with "DaVal" in their keywords get prioritized. Simple, but extremely effective for Korean.

Neural Embedding (multilingual-e5-small)

HuggingFace's multilingual-e5-small model converts text into 384-dimensional vectors. This captures semantic similarity between the real-time question and chunk content.

These three are combined with weights:

최종 스코어 = (Neural × 0.5) + (TF-IDF × 0.2) + (Keyword × 0.3)

Why these ratios? Found through experimentation. Neural gets 50% because semantic similarity matters most. But it's only 50% because over-trusting it on short Korean queries is a mistake. Keyword gets 30% for its strength in exact matching. TF-IDF is a supporting 20%.

If the embedding service goes down? Switch to fallback:

최종 스코어 = (TF-IDF × 0.4) + (Keyword × 0.6)

Since neural is gone, keyword weight is raised to compensate for precision.

Benchmark results: Across 100 test questions, Recall@5 is 93% and MRR (Mean Reciprocal Rank) is 94%. Not bad numbers.

5. Agentic RAG — Decomposing Compound Questions

Say this question comes in: "I'm curious about DaVal's tech stack and Seungwon Jeong's leadership style."

That's actually two different questions. A well-designed system should recognize this, search each one separately, then merge the results.

The trap here: decomposing with an LLM raises costs. So I used rule-based decomposition — no LLM calls.

Logic that recognizes Korean particles:

const query = "기술 스택과 리더십 스타일이 궁금해";
const conjunctions = ["과", "와", "그리고", "또는", "또한"];
 
// "과"를 기준으로 분해
// → ["기술 스택", "리더십 스타일이 궁금해"]

A more refined version handles particle processing. "스택과" → it recognizes "과" as a suffix particle and splits accordingly.

For each decomposed sub-query, a separate search runs. Then:

1. Merge results
2. Remove duplicates
3. Select the top 8 candidates
4. Pass these 8 to LLM Re-ranking

This approach also handles "OR" questions well. "DaVal or SMUSIGNAL?" — both show up in the results.

6. LLM Re-ranking — Improving Precision with Haiku

Hybrid Retrieval produced 8 candidates. But they might not be perfect — a low-relevance result might have slipped in.

So I do Re-ranking with Haiku. Haiku is Claude's lightweight model, roughly 1/5 the cost of Sonnet.

// 8개 후보를 Haiku에 넘김
const prompt = `다음 8개의 문서를 사용자 질문과의 관련도순으로 정렬하세요.
질문: "${userQuery}"
 
${candidates.map(c => `[${c.id}] ${c.content}`).join('\n')}
 
JSON으로 정렬된 순서를 반환하세요:
{"ranking": ["id1", "id2", ...]}`;
 
const response = await claude.messages.create({
  model: "claude-haiku-4-5-20251001",
  messages: [{ role: "user", content: prompt }]
});

But it doesn't apply to every question — cost reasons. When does it activate?

• Compound queries (2 or more sub-queries)
• Long questions (50+ characters)
• Deep conversations (turn 6 or later)

Simple greetings like "Hi!", simple questions like "Who are you?" — re-ranking is skipped.

What if it fails? Fallback to the original order. A re-ranking failure doesn't break the whole response.

7. Cost Optimization — Prompt Caching & Model Routing

This was the part I felt best about. It actually cut costs by 50–60%.

Prompt Caching

I used Anthropic's cache_control feature:

const messages = [
  {
    role: "user",
    content: [
      {
        type: "text",
        text: systemPrompt, // 시스템 프롬프트 (~5,800 토큰)
        cache_control: { type: "ephemeral" }
      },
      {
        type: "text",
        text: `질문별 context와 대화 이력`
      }
    ]
  }
];

The system prompt is nearly identical across all requests. Marking it with cache_control: ephemeral tells Anthropic's servers to cache that section. From the second request onward, input tokens get a 90% discount.

5,800 tokens × 0.9 = 5,220 tokens saved. Per request, that adds up.

Model Routing

I don't use Sonnet for every question. It depends on intent:

const complexIntents = ["technical", "project", "career", "philosophy"];
const simpleIntents = ["greeting", "contact", "troll", "offtopic"];
 
if (complexIntents.includes(intent)) {
  model = "claude-sonnet-4-20250514"; // $3 per 1M input
} else {
  model = "claude-haiku-4-5-20251001"; // $0.80 per 1M input
}

Why would "Hello!" need Sonnet? Haiku is enough. A complex question like "DaVal's architecture and what I learned from it" needs Sonnet.

This alone cuts costs an additional 15–20%.

Other Optimizations

Semantic Cache: If the same question comes in, or a similar question with cosine similarity ≥ 0.92, return the cached response immediately. Skip the LLM call entirely.

Self-Eval Sampling: Not every generated response gets evaluated. 25% sampling. This is for monitoring purposes.

Conversation Compression: After 6 turns, older messages get summarized and stored. Reduces context length while preserving information.

8. Visitor Profiling — Understanding Visitors Without LLM

Here's the interesting part. The same "tech stack" question deserves different answers for a recruiter versus a developer.

For a recruiter: "I mainly use Golang, TypeScript, and PostgreSQL. Golang where performance matters, TypeScript where I need to move fast."

For a developer: dig deeper into the reasoning behind those choices.

So I defined 6 visitor types:

• recruiter: HR or hiring manager
• developer: another developer
• student: student
• founder: startup founder / early team member
• fan: fan (e.g., someone who followed on social media)
• networker: networking purposes

The logic for identifying each type is keyword + pattern based. Not LLM, so cost is zero and it's fast:

const visitorProfile = {
  type: "developer", // 현재 프로파일
  confidence: 0.78,  // 신뢰도
  signals: ["git", "algorithm", "database", "performance"],
  lastUpdate: Date.now()
};
 
// 각 용어가 채팅에 나올 때마다 점수 갱신
if (query.includes("git")) visitorProfile.scores.developer += 0.1;
if (query.includes("채용") || query.includes("면접")) visitorProfile.scores.recruiter += 0.15;

This profile gets injected into the system prompt:

[VISITOR_PROFILE]
Type: developer (confidence: 78%)
Signals: git, algorithm, database

답변 스타일: 기술 깊이 중심, why 강조, trade-off 설명

The LLM reads these signals and automatically adjusts its tone.

9. Prompt Engineering — A Story-First Approach

This is the most important part, and usually the least discussed. "Prompt engineering" is actually engineering. You version-control it like code, test it, and profile it.

The philosophy behind my system prompt:

First, storytelling. Just listing facts comes out stiff.

❌ "정승원은 DaVal의 CTO입니다. DaVal은 2023년 설립된 AI 금융 회사입니다."

✅ "DaVal은 제가 창업한 AI 금융 회사예요. 금융 전문가의 knowledge를 AI로 대중화하려고 만들었어요. 처음엔 작은 아이디어였는데, 지금은..."

Second, Progressive Depth.

• Turn 1: elevator pitch level (30 seconds)
• Turns 2–3: why things were done that way
• Turn 4+: philosophy, doubts, lessons learned

const progressivePrompt = `
현재 턴: ${conversationTurns}
 
${conversationTurns === 1 ? "짧고 friendly한 답변으로 시작" : ""}
${conversationTurns >= 3 ? "deeper insights와 why를 포함" : ""}
${conversationTurns >= 5 ? "philosophical reflection까지 확장 가능" : ""}
`;

Third, Proactive Sharing.

Don't just answer exactly what was asked. Add one more related thought.

Q: "DaVal에서 뭘 배웠어요?"
A: "큼, 데이터 driven decision making의 중요성을 배웠어요.
   특히 금융 도메인에선 가설보다 실증이 중요하다는 걸 깨달았고요.
   그래서 지금 모든 프로덕트 결정에는 A/B test나 실제 유저 피드백이 먼저 와요."

Fourth, language mirroring.

Korean if the user writes in Korean, English if they write in English. Default polite register.

Fifth, Dry Wit.

A tone that's not too stiff — a touch of wit.

"저는 프로덕트를 만드는 걸 좋아하는데,
그게 의료용 로봇이든, 금융 플랫폼이든 간에
결국엔 '사람이 뭘 원하는가'를 이해하는 게 제일 중요하다고 생각해요.
AI가 그걸 도와주는 거고요."

Prompts like these are version-controlled. Saved somewhere like Git, rolled back if performance drops.

10. Feedback Loop — A Self-Improving System

Deploying once isn't the end. It needs to keep improving.

User Feedback

After every response, there are 👍 👎 buttons. Users press 👍 if they liked it, 👎 if not.

Self-Evaluator

25% sampling for automatic evaluation of generated responses. Haiku acts as the judge:

const evaluationCriteria = {
  relevance: "질문과 얼마나 관련이 있는가? (1-5)",
  accuracy: "정보가 정확한가? (1-5)",
  helpfulness: "실제로 도움이 되는가? (1-5)"
};
 
const evaluation = await claude.messages.create({
  model: "claude-haiku-4-5-20251001",
  messages: [{
    role: "user",
    content: `평가: 질문: "${query}", 응답: "${response}", 기준: ${JSON.stringify(evaluationCriteria)}`
  }]
});

Gap Tracker

Questions with a retrieval score below 0.15, or an evaluation score below 3, get logged automatically.

if (retrievalScore < 0.15 || evaluationScore < 3) {
  gaps.push({
    timestamp: Date.now(),
    query,
    score: evaluationScore,
    reason: "low_retrieval" || "low_eval"
  });
}

Gap Monitor

Gap patterns are analyzed every hour. If similar gaps accumulate to 10 or more, an alert is generated:

⚠️ Gap Alert: "DaVal의 go-to-market 전략" 질문이 10번 들어왔는데
모두 retrieval score < 0.15. Knowledge base에 해당 content 추가 필요.

Auto-Learn CLI

Clusters gap data and suggests new knowledge chunks to add:

$ node scripts/suggest-knowledge.js
분석 중...
제안:
1. "DaVal의 GTM 전략" (5개 gap)
2. "why Golang over Python" (3개 gap)
3. "마이크로서비스 vs 모놀리식" (2개 gap)

When I receive these suggestions, I manually write the chunks and add them to the knowledge base.

11. Frontend UX — Real-Time Streaming and Markdown

Streaming

It's SSE (Server-Sent Events) based streaming. Responses appear in real time, token by token.

Users can clearly see that an answer is "being generated." Watching characters appear naturally over 3 seconds is far better than waiting 10 seconds for the whole answer to pop in at once.

Markdown Rendering

marked.js converts markdown to HTML, and DOMPurify prevents XSS:

const html = marked(response);
const sanitized = DOMPurify.sanitize(html);
chatElement.innerHTML = sanitized;

Follow-up Question Skeleton

A "Loading related questions..." skeleton is shown before the response even finishes. It makes the UX feel noticeably smoother.

Visitor Metadata Collection

This is for profiling:

• visitorId: persistent ID stored in localStorage
• device: mobile / tablet / desktop
• timezone: for geographic analysis
• referrer: where they came from

Conversation History Persistence

Conversation history is preserved in localStorage for 24 hours. When someone visits the next day, you can create UX like "Want to pick up where we left off yesterday?"

Rate Limiting (Client)

30-second timeout, 15 requests per minute limit. Prevents users from accidentally clicking multiple times.

12. Operations and Observability — 7 JSONL Logs

A product has to speak through data. So I record 7 types of logs (all in JSONL format):

Analyzing these logs makes it clear where the bottlenecks are and what needs to be improved.

Admin API

Analytics endpoints accessible from the frontend:

GET /api/chat/visitors
// → 오늘의 방문자 분포 (recruiter 30%, developer 40%, ...)
 
GET /api/chat/feedback
// → 평균 만족도, 좋은 평가 비율, 추세
 
GET /api/chat/gaps
// → 최근 갭 alerts, 클러스터, 추천 actions
 
GET /api/chat/latency
// → p50, p99 latency, 각 단계별 breakdown

13. Benchmark Results

Retrieval Performance

Evaluated with 100 test questions:

• Recall@5: 93% (probability that the correct answer is within the top 5)
• MRR (Mean Reciprocal Rank): 94% (how high up the correct answer ranks)

These are excellent numbers.

E2E Response Quality

20 test conversations evaluated by a Haiku judge using criteria-based scoring:

• Overall average: 4.60/5
• Excellent (5 points): 14 (70%)
• Good (4 points): 5 (25%)
• Needs Improvement (≤2 points): 1 (5%)

Breaking it down by category, project, career, fun, contact, and troll handling all scored perfect marks, while philosophy alone came in at 2.0/5 (hit the response length limit). Overall, this is a level you could put in front of a YC startup.

Cost Benchmark

Based on 1,000 requests per month:

• Without prompt caching: ~$8
• With prompt caching: ~$3.5
• With model routing added: ~$3.0

Reduction rate: 62.5%.

14. Tech Stack Summary

15. Conclusion — What I Learned

"Slapping an AI chatbot onto a portfolio" turns out to be deeper than you'd expect.

First, retrieval quality determines everything. Even with a great question coming in, if the system can't find the relevant information, the LLM has no choice but to hallucinate. Flip that around: if retrieval is at 99%, the LLM only needs to perform at 80% for the overall result to exceed 90%.

Second, prompt engineering is code engineering. It needs version control, testing, and CI/CD. It's not about writing "nice-sounding words" — it's about iteratively improving based on measurable performance metrics.

Third, cost optimization is not optional. Prompt caching alone can cut costs by 50%. Don't fall into the trap of "build first, optimize later."

Fourth, graceful degradation is fundamental. Nothing can be perfect all the time. The embedding service might go down; the LLM might be slow. When designing a system, you need to anticipate these failures from the start and build in paths to degrade gracefully.

Next steps: User data is accumulating. In a few months I plan to do custom embedding fine-tuning, which will let me optimize multilingual-e5 for my domain. I'll also keep iterating on response quality continuously.

I think of this not as a "portfolio chatbot," but as a genuinely production-grade AI system.

References & Tools

• Anthropic Claude API (https://www.anthropic.com)
• multilingual-e5-small (https://huggingface.co/intfloat/multilingual-e5-small)
• natural.js (https://github.com/NaturalNode/natural)
• marked.js (https://marked.js.org/)
• DOMPurify (https://github.com/cure53/DOMPurify)

If you have any questions after reading this, ask the AI chatbot on my portfolio site (daemonsw.com) directly. That's the best way to see every system described in this post actually working.