SWE-chat: Dataset Card

SWE-chat: Coding Agent Interactions From Real Users in the Wild

• Paper: arxiv.org/abs/2604.20779
• Website: swe-chat.com

Dataset Summary

SWE-chat captures real-world AI coding sessions from developers using AI coding assistants (Claude Code, Codex, Gemini CLI, and others via the Entire.io CLI). Each session includes the full conversation transcript, tool calls, thinking traces, code changes, and attribution of human vs. agent-authored code.

Dataset Size

Tables

Loading the Data

from datasets import load_dataset

# Load individual tables (each table is exposed as a named config)
conversations = load_dataset("SALT-NLP/SWE-chat", "conversations", split="train")
sessions = load_dataset("SALT-NLP/SWE-chat", "sessions", split="train")
commits = load_dataset("SALT-NLP/SWE-chat", "commits", split="train")

# Or load with pandas directly
import pandas as pd
df_conv = pd.read_parquet("conversations.parquet")
df_sessions = pd.read_parquet("sessions.parquet")

# Raw transcripts live under transcripts/{session_id}.jsonl
# session_logs.parquet holds the relative path + auxiliary text fields
# Compacted transcripts: transcripts_compact/{session_id}.jsonl
# Subagent transcripts: transcripts_subagent/{agent_id}__{tool_use_id}.jsonl
df_session_logs = pd.read_parquet("session_logs.parquet")
with open(df_session_logs["transcript_path"].iloc[0]) as f:
    raw_transcript = f.read()

Quick Examples

import pandas as pd

conversations = pd.read_parquet("conversations.parquet")
sessions = pd.read_parquet("sessions.parquet")

# Compact user-assistant conversation (no tool calls/thinking)
conv = conversations[conversations.is_conversational]
print(conv[["conversation_turn_number", "role", "content"]].head(10))

# All user prompts
prompts = conversations[conversations.turn_type == 'user_prompt']
print(prompts[["content", "agent", "language", "prompt_intent", "prompt_pushback"]])


# All tool calls with extracted file paths
tools = conversations[conversations.turn_type == "tool_use"]
print(tools[["tool_name", "file_path", "command"]].head())

# Recorded parent-side subagent calls (inspect JSON arguments for task text).
delegations = conversations.loc[(conversations.turn_type == "tool_use") & (conversations.orchestration_kind == "subagent_spawn")]
print(delegations[["session_id", "turn_id", "tool_name", "tool_input_json"]].head().to_string(index=False))

# Agent thinking traces
thinking = conversations[conversations.turn_type == "assistant_thinking"]
print(thinking[thinking['content']!=''][["content", "agent", "model"]])

# Sessions with high agent-authored code percentage
high_agent = sessions[sessions.agent_percentage > 90]

# Join conversations to sessions for richer context
merged = conversations.merge(
    sessions[["session_id", "repo_id", "agent_percentage"]],
    on="session_id"
)

Dataset Schema

Relationship Map

repositories (1) ───-> (N) checkpoints    [repo_id]
checkpoints (N) ←──-> (M) sessions        [JSON arrays on both sides]
checkpoints (1) ───-> (N) commits          [checkpoint_pk]
sessions (1) ───-> (1) session_logs        [session_id]
sessions (1) ───-> (N) conversations       [session_id]
checkpoints (1) ───-> (N) subagent_tasks   [checkpoint_pk]
subagent_tasks (1) ───-> (N) conversations_subagents  [task_pk]
skill_definitions (1) ───-> (N) skill_invocations     [definition_id]
sessions (1) ───-> (N) skill_invocations/context_events  [session_id, when available]

subagent_tasks.session_id and conversations_subagents.session_id hold the parent session's id, since a subagent runs inside its parent's session. That is why subagent turns live in their own table: keyed on session_id they would collide with the parent's rows in conversations.

ID Strategy

turn_id is unique within a release, but not stable across releases. turn_number is a positional index over all rows, which can change if sessions update across release. Join on session_id (stable) rather than turn_id when comparing across releases.

---

1. conversations.parquet

One row per conversation turn (user prompt, assistant response, assistant thinking step, tool call, or metadata event). Filter with is_conversational=True for a clean user-assistant dialogue view.

Two turn columns: - turn_number: sequential across ALL rows in the session (including tool calls) - conversation_turn_number: sequential ONLY for is_conversational=True rows. NULL for tool/thinking rows

Row types produced from transcript entries

Note on queue_operation rows: Claude Code queues messages when the agent is busy. The queue_op_subtype column distinguishes: (a) user prompts typed while agent was running (user_prompt_enqueued), which may have been delivered (user_prompt_delivered) or silently discarded before reaching the agent (user_prompt_discarded); and (b) completed subagent results (task_notification). The content column contains the plain-text prompt or the <task-notification> XML payload; it is empty for dequeue and remove operations.

2. sessions.parquet

Note: Session token and attribution fields come from metadata recorded by Entire from the agent integration. They are not reconstructed by summing parsed tool calls. By contrast, conversation-row usage is parsed from native transcript usage records, and git diff/history totals are computed locally.

2b. session_logs.parquet

One row per session. The raw JSONL/JSON transcript is stored as a file under transcripts/{session_id}.jsonl (or .json for Gemini CLI) and referenced by the transcript_path column. Join to sessions on session_id.

Some sessions additionally have a compact transcript under transcripts_compact/{session_id}.jsonl (compact_transcript_path). Prefer transcripts/ for analysis — it is the agent's own complete record; the compact form is derived and lossy (no system turns or reasoning, and parallel tool results may be missing).

3. checkpoints.parquet

4. commits.parquet

5. repositories.parquet

6. subagent_tasks.parquet

Captured subagent task records (non-exhaustive list of delegations). Join to sessions on session_id — which is the parent session's id — or to checkpoints on checkpoint_pk.

Older CLI versions recorded only a subagent's identity and its link to the parent session; newer ones also store the subagent's own transcript. Columns that only the newer format supplies are empty for the older rows, and such a task has no rows in conversations_subagents. For broader historical coverage, select turn_type == "tool_use" and orchestration_kind == "subagent_spawn" from conversations.

7. conversations_subagents.parquet

Turns from subagent transcripts. Same idea as conversations but for work done by a subagent, and deliberately a separate table: these rows carry the parent session's session_id, so merging them into conversations would collide with the parent's own turns.

Only available subagent transcripts contribute rows here. A child transcript's user role can contain instructions supplied by the parent agent or harness.

8. skill_definitions.parquet

9. skill_invocations.parquet

10. context_events.parquet

Data Collection

Source

Data is collected from public GitHub repositories that use the Entire.io CLI to checkpoint their AI coding sessions. Checkpoints are stored either on a special branch ()entire/checkpoints/v1) or in one ref per checkpoint under refs/entire/checkpoints/, containing:

• Session metadata (agent, strategy, token usage, code attribution)
• Full conversation transcripts (JSONL for Claude Code, JSON for Gemini CLI, etc.)
• User prompts and context

Supported Agents

• Claude Code
• OpenAI Codex
• Gemini CLI
• Cursor
• OpenCode
• GitHub Copilot CLI
• Factory AI Droid
• Pi

PII Redaction

We redacted personally identifiable information in all user prompts, assistant responses, subagent prose/task descriptions, and released skills using Microsoft Presidio (named-entity detection) and TruffleHog (secret detection).

Deduplication Strategy

Sessions may appear in multiple checkpoints (a session spans checkpoint boundaries). We deduplicate on session_id, keeping the record with the highest output_tokens (most complete). The checkpoint_ids column preserves the full list of checkpoints each session appeared in.

Data Removal Requests

If you would like your data removed from SWE-chat, or if you encounter content that is illegal, you may request deletion. To do so, please contact us via [email protected]. Please include repo_id, session_id, or turn_id values corresponding to the entries you wish to remove, and a brief explanation of the reason for removal.

SWE-chat dataset versions

Citation

Please consider citing the following papers if you find this dataset useful:

@inproceedings{baumann2026swechat,
  title={SWE-chat: Real-World AI Coding Sessions in the Wild},
  author={Baumann, Joachim and Padmakumar, Vishakh and Li, Xiang and Yang, John and Yang, Diyi and Koyejo, Sanmi},
  booktitle={Third Conference on Language Modeling},
  year={2026},
  url={https://arxiv.org/pdf/2604.20779}
}