Skip to main content

Real-time Updates (SSE)

Server-Sent Events (SSE) provide real-time progress updates during report generation. This is the recommended way to monitor report creation status.

Overview​

SSE is a one-way communication channel from server to client, perfect for:

  • Monitoring profile data loading progress
  • Tracking AI summary generation
  • Receiving completion notifications
  • Keeping connections alive with heartbeat pings

Connecting to SSE​

Endpoint​

GET /sse/{reportId}

Authentication and access​

A Bearer token is optional, but the report must be readable by the caller: the report's owner, or the guest session that owns a report nobody has claimed yet. A caller who may not read the report gets 403 {"code": 403, "message": "access denied"}; an unknown report ID gets 404.

curl -N https://dashboard.socialprofiler.com/api/v1/sse/68af01be012b843fd7bc2bf4 \
-H "Authorization: Bearer YOUR_TOKEN_HERE" \
-H "Accept: text/event-stream"

A browser EventSource cannot send an Authorization header: use the session cookie, or a fetch-based SSE client (see the examples below).

Response Headers​

Content-Type: text/event-stream
Cache-Control: no-cache
Connection: keep-alive
Transfer-Encoding: chunked
X-Accel-Buffering: no

X-Accel-Buffering: no asks proxies not to buffer the stream, so events arrive as they happen.

Event Types​

Ping Events​

The first ping arrives shortly after the connection opens with status: "inited"; subsequent keepalive pings use status: "inprogress" and repeat every 15 seconds by default (server-configured). A ping's status is only ever inited or inprogress:

event: ping
data: {"type":"ping","status":"inited"}
event: ping
data: {"type":"ping","status":"inprogress"}

Follows Events​

Profile data loading progress:

event: follows
data: {"type":"follows","status":"inprogress","id":"123456789","source":"ig"}
event: follows
data: {"type":"follows","status":"done","id":"123456789","source":"ig"}
event: follows
data: {"type":"follows","status":"error","id":"123456789","source":"ig","error":"private"}

Reconnect snapshot​

The stream keeps no replay, so a follows result broadcast while the browser was reconnecting is lost. A (re)connecting client therefore receives, to itself only and right after it is registered, one follows message per source whose on-demand load is terminal: error with the persisted reason when the source has a failure recorded for its account, done when the load was requested and has finished. A source still loading, or never loaded on demand, sends nothing. The messages are derived from the persisted report state alone, not from the last live event. The reason is the stored failure with any text after ": " cut off; a known reason (private, unmapped, notFound, optOut, zeroFollows, followsPrivate, unloaded, unexpected) is kept as it is. A raw failure text from the loader is reported as the reason the live event would carry (empty as zeroFollows, notfound as notFound, private as private, unmapped as unmapped, anything unrecognised as unexpected).

Two rules keep the snapshot and the live stream in step:

  • A success that arrives after the source was settled as failed for the same account (the 15-minute ceiling stored unexpected, or a failure was recorded) updates the report to loaded and removes the error, but only while the report is not generated yet (no tracked, and its generation did not succeed). A generated report is left as it was, and the snapshot says error.
  • A final failure after a preview that delivered mapped interests counts as success (Instagram, Twitter, Facebook; a preview whose follows mapped to nothing settles no report and does not count): the report is already built from the preview, so nothing is written as an error and the live event is done. The task's own status still records the failure, and the failure's profile flags are not written. TikTok is not covered: a TikTok failure after a preview stays an error.
  • TikTok follows whose handle now belongs to another account (the scraper answers IDMISMATCH): the backend re-resolves it per report. A report switched to the handle's current owner has its source rekeyed, and its follows event is sent under the new id and again under the old one, so a page opened before the switch still sees it. A mismatch that cannot be resolved (the handle's owner is gone, or switching is off) ends the source with reason notFound.

Summary Events​

AI summary generation signal. The event never carries the summary itself: done means the summary is ready, and the client then fetches it with GET /report/{reportId}/summary, which applies the viewer's own paywall and blur rules.

event: summary
data: {"type":"summary","status":"done"}

On failure the error text is always generic:

event: summary
data: {"type":"summary","status":"error","error":"unexpected error"}

Stream Lifetime​

Events are broadcast to every client connected to the report, so all viewers receive the same payloads. A done or error event does not end the stream: the connection stays open, kept alive by pings, until the client disconnects. Close the connection yourself once you have what you need.

Confirm Events​

The confirm event reports that the user's email address was confirmed (the user opened the link from the confirmation email, POST /auth/confirm/{hash}). It is not about choosing accounts for the report. It carries no account: only the type and status. It is sent to every open report stream of that user:

event: confirm
data: {"type":"confirm","status":"done"}

Status Values​

StatusDescription
initedTask initialized, waiting to start
inprogressTask is running
doneTask completed successfully
errorTask failed (check error field)

Error Reasons​

When status: "error", the error field contains:

ErrorDescription
privateProfile is private
notFoundProfile doesn't exist
zeroFollowsProfile has no follows
followsPrivateTikTok hid the following list (the account itself is not private)
optOutUser opted out of tracking
unloadedData couldn't be loaded
unmappedProfile couldn't be mapped
unexpectedUnexpected failure

Client Implementation​

JavaScript (Browser)​

A browser EventSource cannot set headers, so it authenticates with the session cookie (withCredentials). To send a Bearer token from a browser, use a fetch-based SSE client such as @microsoft/fetch-event-source. Every event is named (ping, follows, summary, confirm), so register listeners with addEventListener('<name>'): onmessage never fires.

function connectSSE(reportId) {
const url = `https://dashboard.socialprofiler.com/api/v1/sse/${reportId}`;

// Uses the session cookie set at sign-in.
const eventSource = new EventSource(url, { withCredentials: true });

// Handle ping events
eventSource.addEventListener('ping', (event) => {
const data = JSON.parse(event.data);
console.log('Ping:', data.status);
});

// Handle follows loading progress
eventSource.addEventListener('follows', (event) => {
const data = JSON.parse(event.data);
console.log(`Profile ${data.id} (${data.source}): ${data.status}`);

if (data.status === 'error') {
console.error(`Error: ${data.error}`);
}
});

// Handle summary generation
eventSource.addEventListener('summary', (event) => {
const data = JSON.parse(event.data);
console.log(`Summary: ${data.status}`);

if (data.status === 'done') {
// The event is a signal only: fetch GET /report/{reportId}/summary for the content.
console.log('Summary ready!');
eventSource.close();
}
});

// Handle errors
eventSource.onerror = (error) => {
console.error('SSE Error:', error);
eventSource.close();
};

return eventSource;
}

// Usage
const sse = connectSSE('68af01be012b843fd7bc2bf4');

// Close when done
// sse.close();

JavaScript (Node.js)​

Using the eventsource package, which lets you set headers:

const EventSource = require('eventsource');

function connectSSE(reportId, token) {
const url = `https://dashboard.socialprofiler.com/api/v1/sse/${reportId}`;

const eventSource = new EventSource(url, {
headers: {
'Authorization': `Bearer ${token}`
}
});

// Events are named, so onmessage never fires: listen for each name.
for (const name of ['ping', 'follows', 'summary', 'confirm']) {
eventSource.addEventListener(name, (event) => {
console.log(name, JSON.parse(event.data));
});
}

eventSource.onerror = (error) => {
console.error('Error:', error);
};

return eventSource;
}

Python​

Using the sseclient-py package:

import json

import sseclient
import requests

def connect_sse(report_id: str, token: str):
url = f'https://dashboard.socialprofiler.com/api/v1/sse/{report_id}'
headers = {
'Authorization': f'Bearer {token}',
'Accept': 'text/event-stream'
}

response = requests.get(url, headers=headers, stream=True)
client = sseclient.SSEClient(response)

for event in client.events():
data = json.loads(event.data)
event_type = event.event

print(f'{event_type}: {data}')

if event_type == 'summary' and data['status'] == 'done':
# The event is a signal only: fetch the summary itself.
summary = requests.get(
f'https://dashboard.socialprofiler.com/api/v1/report/{report_id}/summary',
headers={'Authorization': f'Bearer {token}'},
).json()
print('Report ready!', summary)
break

# Usage
connect_sse('68af01be012b843fd7bc2bf4', 'your_token')

Go​

package main

import (
"bufio"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
)

type SSEMessage struct {
Type string `json:"type"`
Status string `json:"status"`
ID string `json:"id,omitempty"`
Source string `json:"source,omitempty"`
Error string `json:"error,omitempty"`
}

func connectSSE(reportID, token string) error {
url := fmt.Sprintf("https://dashboard.socialprofiler.com/api/v1/sse/%s", reportID)

req, _ := http.NewRequest("GET", url, nil)
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Accept", "text/event-stream")

resp, err := http.DefaultClient.Do(req)
if err != nil {
return err
}
defer resp.Body.Close()

scanner := bufio.NewScanner(resp.Body)
var eventType string

for scanner.Scan() {
line := scanner.Text()

if strings.HasPrefix(line, "event:") {
eventType = strings.TrimSpace(strings.TrimPrefix(line, "event:"))
} else if strings.HasPrefix(line, "data:") {
data := strings.TrimPrefix(line, "data:")
var msg SSEMessage
json.Unmarshal([]byte(data), &msg)

fmt.Printf("%s: %+v\n", eventType, msg)

if eventType == "summary" && msg.Status == "done" {
// The event is a signal only: fetch the summary itself.
sreq, err := http.NewRequest("GET", fmt.Sprintf("https://dashboard.socialprofiler.com/api/v1/report/%s/summary", reportID), nil)
if err != nil {
return err
}
sreq.Header.Set("Authorization", "Bearer "+token)
sresp, err := http.DefaultClient.Do(sreq)
if err != nil {
return err
}
body, err := io.ReadAll(sresp.Body)
sresp.Body.Close()
if err != nil {
return err
}
if sresp.StatusCode != http.StatusOK {
fmt.Println("Summary request failed:", sresp.Status)
return nil
}
fmt.Println("Report ready!", string(body))
return nil
}
}
}

return scanner.Err()
}

Best Practices​

1. Handle Reconnection​

SSE connections can drop. Implement automatic reconnection:

function connectWithRetry(reportId, maxRetries = 3) {
let retries = 0;

function connect() {
const sse = connectSSE(reportId);

sse.onerror = () => {
sse.close();
if (retries < maxRetries) {
retries++;
console.log(`Reconnecting... (attempt ${retries})`);
setTimeout(connect, 1000 * retries);
}
};

return sse;
}

return connect();
}

2. Set Timeouts​

Don't wait forever for events:

const timeout = setTimeout(() => {
sse.close();
console.log('Timeout - checking status via API');
}, 120000); // 2 minutes

sse.addEventListener('summary', (event) => {
if (JSON.parse(event.data).status === 'done') {
clearTimeout(timeout);
}
});

3. Track Progress​

Keep track of which profiles are loaded:

const profileStatus = {};

sse.addEventListener('follows', (event) => {
const data = JSON.parse(event.data);
profileStatus[`${data.source}:${data.id}`] = data.status;

const total = Object.keys(profileStatus).length;
const done = Object.values(profileStatus).filter(s => s === 'done').length;
console.log(`Progress: ${done}/${total}`);
});

4. Graceful Degradation​

Fall back to polling if SSE fails:

async function waitForReport(reportId, token) {
try {
await new Promise((resolve, reject) => {
const sse = connectSSE(reportId);
sse.addEventListener('summary', (e) => {
if (JSON.parse(e.data).status === 'done') {
sse.close();
resolve();
}
});
sse.onerror = reject;
});
} catch (error) {
console.log('SSE failed, falling back to polling');
await pollForCompletion(reportId, token);
}
}

async function pollForCompletion(reportId, token) {
while (true) {
const response = await fetch(`/api/v1/check/${reportId}`, {
headers: { 'Authorization': `Bearer ${token}` }
});
const data = await response.json();
if (data.ready) break;
await new Promise(r => setTimeout(r, 5000));
}
}