# Build a Custom Report

> Turn exported Copyleaks scan results into your own report - read the completed webhook, export each result and the crawled version, and map match positions to highlights.

When the Hosted Web Report and the Open-Source Web Report don't fit your product, you can build your own report from the raw scan data. Copyleaks gives you the full text of the scanned document, the text of every matched source, and the exact positions of each match, so you can draw the highlights and the side-by-side comparison in your own interface.

This guide picks up after a scan has completed. It walks through collecting the results, exporting the data, and turning match positions into highlights.

## Get started

<Steps>
  <Step title="Before you begin">
    Before you begin, ensure you have:
    - A scan submitted with a status webhook, as shown in the [Detect Plagiarism in Text](/guides/authenticity/detect-plagiarism-text) guide. That guide also covers installing an SDK and logging in.
    - Public endpoints on your server (or signed storage URLs) that Copyleaks can upload the exported files to.

    Some data is only produced when you ask for it in the scan submission:
    - `properties.pdf.create: true` if you want to offer the PDF report.
    - `properties.aiGeneratedText.detect: true` if you want to show AI Text Detection results.
    - `properties.includeHtml: true` if you want to render the HTML version of the document. This guide uses the text version, which every scan includes.
  </Step>

  <Step title="Read the completed webhook">
    When the scan finishes, Copyleaks sends the [completed webhook](/reference/data-types/authenticity/webhooks/scan-completed) to your status URL. It is the summary of your report: it lists every matched source, grouped by where it was found, and the totals for the whole document.

    ```json
    {
      "scannedDocument": {
        "scanId": "my-scan-1",
        "totalWords": 350
      },
      "results": {
        "internet": [
          {
            "id": "2a1b402420",
            "title": "Lions - Wikipedia",
            "introduction": "The lion is a large cat of the genus Panthera...",
            "matchedWords": 42,
            "url": "https://en.wikipedia.org/wiki/Lion"
          }
        ],
        "database": [],
        "batch": [],
        "repositories": [],
        "internalAIData": [],
        "score": {
          "identicalWords": 30,
          "minorChangedWords": 8,
          "relatedMeaningWords": 4,
          "aggregatedScore": 12
        }
      }
    }
    ```

    - `results.internet`, `results.database` (the Shared Data Hub), `results.batch`, `results.repositories` (your Private Cloud Hubs) and `results.internalAIData` each hold a list of matched sources. Use `title`, `introduction`, `matchedWords` and, for internet results, `url` to build the list of sources in your report.
    - Keep the `id` of every result. You need it to export that result's full text and match positions.
    - `results.score` holds the totals for the document: the number of identical, minor-change and related-meaning (paraphrased) words, and the `aggregatedScore`.
    - `notifications.alerts` lists any [scan alerts](/reference/data-types/authenticity/scan-alerts) you may want to show alongside the report.
  </Step>

  <Step title="Export the results and the crawled version">
    The completed webhook only summarizes the matches. To draw highlights you need two kinds of files, which you get from the [Export](/reference/actions/downloads/export) endpoint:
    - The [crawled version](/reference/data-types/authenticity/results/crawled-version): the text of the scanned document, as Copyleaks read it.
    - One [result](/reference/data-types/authenticity/results/new-plagiarism-result) per matched source: the source's text and the positions of every match.

    Request them all in one export call. Choose a new `exportId` for each export, and pass up to 1,000 result IDs from the completed webhook.

    <CodeGroup>
      ```http title="HTTP" icon="globe"
      POST https://api.copyleaks.com/v3/downloads/my-scan-1/export/my-export-1

      Headers
      Authorization: Bearer <YOUR_AUTH_TOKEN>
      Content-Type: application/json

      Body
      {
        "completionWebhook": "https://your-server.com/webhook/export/my-export-1/completed",
        "maxRetries": 3,
        "crawledVersion": {
          "endpoint": "https://your-server.com/webhook/export/my-export-1/crawled-version",
          "verb": "POST"
        },
        "results": [
          {
            "id": "2a1b402420",
            "endpoint": "https://your-server.com/webhook/export/my-export-1/results/2a1b402420",
            "verb": "POST"
          }
        ]
      }
      ```
      ```bash title="cURL" icon="terminal"
      curl -X POST "https://api.copyleaks.com/v3/downloads/my-scan-1/export/my-export-1" \
           -H "Authorization: Bearer <YOUR_AUTH_TOKEN>" \
           -H "Content-Type: application/json" \
           -d '{
                 "completionWebhook": "https://your-server.com/webhook/export/my-export-1/completed",
                 "maxRetries": 3,
                 "crawledVersion": {
                   "endpoint": "https://your-server.com/webhook/export/my-export-1/crawled-version",
                   "verb": "POST"
                 },
                 "results": [
                   {
                     "id": "2a1b402420",
                     "endpoint": "https://your-server.com/webhook/export/my-export-1/results/2a1b402420",
                     "verb": "POST"
                   }
                 ]
               }'
      ```
      ```python title="Python" icon="python"
      from copyleaks.copyleaks import Copyleaks
      from copyleaks.models.export import Export, ExportResult, ExportCrawledVersion

      auth_token = Copyleaks.login("your@email.address", "YOUR_API_KEY")

      scan_id = "my-scan-1"
      export_id = "my-export-1"
      base_url = f"https://your-server.com/webhook/export/{export_id}"
      result_ids = ["2a1b402420"]  # every result ID from the completed webhook

      export = Export()
      export.set_completion_webhook(f"{base_url}/completed")

      crawled = ExportCrawledVersion()
      crawled.set_endpoint(f"{base_url}/crawled-version")
      crawled.set_verb("POST")
      export.set_crawled_version(crawled)

      results = []
      for result_id in result_ids:
          result = ExportResult()
          result.set_id(result_id)
          result.set_endpoint(f"{base_url}/results/{result_id}")
          result.set_verb("POST")
          results.append(result)
      export.set_results(results)

      Copyleaks.export(auth_token, scan_id, export_id, export)
      ```
      ```javascript title="JavaScript" icon="square-js"
      const { Copyleaks, CopyleaksExportModel } = require('copyleaks');

      async function exportForReport() {
        const copyleaks = new Copyleaks();
        const loginResult = await copyleaks.loginAsync('YOUR_EMAIL@example.com', 'YOUR_API_KEY');

        const scanId = 'my-scan-1';
        const exportId = 'my-export-1';
        const baseUrl = `https://your-server.com/webhook/export/${exportId}`;
        const resultIds = ['2a1b402420']; // every result ID from the completed webhook

        const exportModel = new CopyleaksExportModel(
          `${baseUrl}/completed`,
          resultIds.map(id => ({ id, endpoint: `${baseUrl}/results/${id}`, verb: 'POST' })),
          { endpoint: `${baseUrl}/crawled-version`, verb: 'POST' }
        );

        await copyleaks.exportAsync(loginResult, scanId, exportId, exportModel);
      }

      exportForReport();
      ```
      ```java title="Java" icon="java"
      import classes.Copyleaks;
      import models.exports.*;
      import models.response.CopyleaksAuthToken;

      CopyleaksAuthToken authToken = Copyleaks.login("your@email.address", "YOUR_API_KEY");

      String scanId = "my-scan-1";
      String exportId = "my-export-1";
      String baseUrl = "https://your-server.com/webhook/export/" + exportId;
      String[] resultIds = { "2a1b402420" }; // every result ID from the completed webhook

      ExportResults[] results = new ExportResults[resultIds.length];
      for (int i = 0; i < resultIds.length; i++) {
          results[i] = new ExportResults(resultIds[i], baseUrl + "/results/" + resultIds[i], "POST", null);
      }

      ExportCrawledVersion crawledVersion = new ExportCrawledVersion(baseUrl + "/crawled-version", "POST", null);

      CopyleaksExportModel exportModel = new CopyleaksExportModel(baseUrl + "/completed", results, crawledVersion);
      Copyleaks.export(authToken, scanId, exportId, exportModel);
      ```
    </CodeGroup>

    <Tip>
    Include the result ID in each endpoint URL, as above, so you can tell the uploaded results apart when they arrive. To protect your endpoints, add your own authorization header to each item with its `headers` field, an array of name and value pairs such as `"headers": [["Authorization", "Bearer <your-secret>"]]`. Protect the completion webhook the same way with `completionWebhookHeaders`. If you upload to signed storage URLs, use `PUT` as the verb, so a repeated upload can overwrite the file.
    </Tip>
  </Step>

  <Step title="Receive the exported files">
    Copyleaks uploads each file to the endpoint you gave it, using the verb you chose. The crawled version and the results are JSON. Store them, keyed by scan ID and result ID, so your report can load them later.

    When every upload is done, Copyleaks calls your `completionWebhook` with the [Export Completed](/reference/data-types/authenticity/webhooks/export-completed) payload. Check that `completed` is `true` and that every entry in `tasks` has `isHealthy: true` before you show the report. If a task failed, run a new export with a new `exportId` for the missing files.
  </Step>

  <Step title="Map match positions to highlights">
    Each result holds the matched source's text in `text.value`, and the match positions in `text.comparison`. Matches are split into three types, from strongest to weakest:

    | Type | Meaning |
    | :--- | :--- |
    | `identical` | Exactly the same words. |
    | `minorChanges` | Nearly identical words with small differences, like "slow" and "slowly". |
    | `relatedMeaning` | Paraphrased text that states a similar idea in different words. |

    Every type has two sides, and each side lists the matches as parallel `starts` and `lengths` arrays:
    - `source` positions point into the scanned document: the crawled version's `text.value`.
    - `suspected` positions point into the matched source: this result's own `text.value`.

    `chars` holds character positions, which you use to slice the text. `words` holds the same matches as word positions.

    Character positions count UTF-16 code units, the way JavaScript, Java and C# index strings. A character outside the Basic Multilingual Plane, such as an emoji or a math letter like 𝑥, counts as two. Python indexes strings by code point instead, so in Python, slice the UTF-16 encoding of the text, as the example below does. Otherwise every highlight after such a character lands in the wrong place.

    ```json
    "identical": {
      "source":    { "chars": { "starts": [0], "lengths": [20] } },
      "suspected": { "chars": { "starts": [13], "lengths": [20] } }
    }
    ```

    The `n`th match runs from `starts[n]` for `lengths[n]` characters. Here, characters 0 to 19 of the scanned document match characters 13 to 32 of the matched source.

    The functions below collect the ranges for one side of a result and split a text into plain and highlighted segments. Several results can match the same passage of the scanned document, so where ranges overlap, the strongest type wins.

    <CodeGroup>
      ```javascript title="JavaScript" icon="square-js"
      // Where matches overlap, the strongest type wins.
      const PRIORITY = { identical: 3, minorChanges: 2, relatedMeaning: 1 };

      // side "source" points into the crawled version's text.value,
      // side "suspected" into the result's own text.value.
      function collectRanges(result, side) {
        const ranges = [];
        for (const type of Object.keys(PRIORITY)) {
          const { starts, lengths } = result.text.comparison[type][side].chars;
          starts.forEach((start, i) => ranges.push({ start, end: start + lengths[i], type }));
        }
        return ranges;
      }

      // Split the text into runs that share the same highlight (or none).
      function toSegments(text, ranges) {
        const marks = new Array(text.length).fill(null);
        for (const { start, end, type } of ranges) {
          for (let i = start; i < Math.min(end, text.length); i++) {
            if (!marks[i] || (PRIORITY[type] ?? 0) > (PRIORITY[marks[i]] ?? 0)) marks[i] = type;
          }
        }
        const segments = [];
        let from = 0;
        for (let i = 1; i <= text.length; i++) {
          if (i === text.length || marks[i] !== marks[from]) {
            segments.push({ text: text.slice(from, i), type: marks[from] });
            from = i;
          }
        }
        return segments;
      }
      ```
      ```python title="Python" icon="python"
      # Where matches overlap, the strongest type wins.
      PRIORITY = {"identical": 3, "minorChanges": 2, "relatedMeaning": 1}


      def collect_ranges(result, side):
          """side="source" points into the crawled version's text.value,
          side="suspected" into the result's own text.value."""
          ranges = []
          for match_type in PRIORITY:
              chars = result["text"]["comparison"][match_type][side]["chars"]
              for start, length in zip(chars["starts"], chars["lengths"]):
                  ranges.append((start, start + length, match_type))
          return ranges


      def to_segments(text, ranges):
          """Split the text into runs that share the same highlight (or none).
          Positions count UTF-16 code units, so work on the UTF-16 encoding of the text."""
          units = text.encode("utf-16-le", "surrogatepass")
          size = len(units) // 2
          marks = [None] * size
          for start, end, match_type in ranges:
              for i in range(start, min(end, size)):
                  if marks[i] is None or PRIORITY.get(match_type, 0) > PRIORITY.get(marks[i], 0):
                      marks[i] = match_type
          segments, begin = [], 0
          for i in range(1, size + 1):
              if i == size or marks[i] != marks[begin]:
                  segments.append((units[2 * begin:2 * i].decode("utf-16-le", "surrogatepass"), marks[begin]))
                  begin = i
          return segments
      ```
      ```java title="Java" icon="java"
      import com.google.gson.JsonArray;
      import com.google.gson.JsonObject;
      import java.util.ArrayList;
      import java.util.List;
      import java.util.Map;
      import java.util.Objects;

      // Uses records, so it needs Java 16 or later.
      public class ReportHighlighter {
          // Where matches overlap, the strongest type wins.
          static final Map<String, Integer> PRIORITY =
                  Map.of("identical", 3, "minorChanges", 2, "relatedMeaning", 1);

          record Range(int start, int end, String type) {}
          record Segment(String text, String type) {}

          // side "source" points into the crawled version's text.value,
          // side "suspected" into the result's own text.value.
          static List<Range> collectRanges(JsonObject result, String side) {
              List<Range> ranges = new ArrayList<>();
              JsonObject comparison = result.getAsJsonObject("text").getAsJsonObject("comparison");
              for (String type : PRIORITY.keySet()) {
                  JsonObject chars = comparison.getAsJsonObject(type).getAsJsonObject(side).getAsJsonObject("chars");
                  JsonArray starts = chars.getAsJsonArray("starts");
                  JsonArray lengths = chars.getAsJsonArray("lengths");
                  for (int i = 0; i < starts.size(); i++) {
                      int start = starts.get(i).getAsInt();
                      ranges.add(new Range(start, start + lengths.get(i).getAsInt(), type));
                  }
              }
              return ranges;
          }

          // Split the text into runs that share the same highlight (or none).
          static List<Segment> toSegments(String text, List<Range> ranges) {
              String[] marks = new String[text.length()];
              for (Range r : ranges) {
                  for (int i = r.start(); i < Math.min(r.end(), text.length()); i++) {
                      if (marks[i] == null
                              || PRIORITY.getOrDefault(r.type(), 0) > PRIORITY.getOrDefault(marks[i], 0)) {
                          marks[i] = r.type();
                      }
                  }
              }
              List<Segment> segments = new ArrayList<>();
              int from = 0;
              for (int i = 1; i <= text.length(); i++) {
                  if (i == text.length() || !Objects.equals(marks[i], marks[from])) {
                      segments.add(new Segment(text.substring(from, i), marks[from]));
                      from = i;
                  }
              }
              return segments;
          }
      }
      ```
    </CodeGroup>

    Use them for the two views most reports have:
    - **Document view**: collect the `source` ranges of every result and apply them to the crawled version's `text.value`. This shows every matched passage in the scanned document.
    - **Comparison view**: for one result, apply its `source` ranges to the crawled version's `text.value` and its `suspected` ranges to the result's own `text.value`, and show the two side by side.

    The text comes from the scanned document and from third-party sources, so treat it as untrusted. In a browser, build the highlights with `textContent` instead of inserting HTML:

    ```javascript title="Render in the browser"
    function render(container, segments) {
      for (const { text, type } of segments) {
        if (!type) {
          container.appendChild(document.createTextNode(text));
          continue;
        }
        const mark = document.createElement('mark');
        mark.className = type; // style .identical, .minorChanges and .relatedMeaning in your CSS
        mark.textContent = text;
        container.appendChild(mark);
      }
    }

    // Document view: every result's matches over the scanned document.
    // crawledVersion and resultsById (result ID -> result) are the files you stored earlier.
    const ranges = Object.values(resultsById).flatMap(result => collectRanges(result, 'source'));
    render(document.getElementById('document-view'), toSegments(crawledVersion.text.value, ranges));
    ```

    <Note>
    Some results come back masked. Shared Data Hub results from documents another user submitted are always masked, and Private Cloud Hub results are masked depending on the hub's [masking policy](/reference/data-types/authenticity/private-cloud-hub/masking-policy). In a masked result only the identical passages stay readable: the rest of the text, including minor-change and related-meaning passages, shows `#` in place of each character other than spaces, line breaks and basic punctuation. The replacement is one for one, so the positions still line up with the text.
    </Note>

    The crawled version also lists the parts of the document that were excluded from the scan in `text.exclude`, for example quotes or references excluded by your `properties.exclude` settings, and the page boundaries in `text.pages.startPosition`. You can use them the same way, for example to grey out excluded text or to split the document into pages. See the [Crawled Version](/reference/data-types/authenticity/results/crawled-version) reference.
  </Step>

  <Step title="Add AI Text Detection results (optional)">
    If the scan was submitted with `properties.aiGeneratedText.detect: true`, add an `aiDetection` item to the same export request:

    ```json
    "aiDetection": {
      "endpoint": "https://your-server.com/webhook/export/my-export-1/ai-detection",
      "verb": "POST"
    }
    ```

    With the Java SDK, call `exportModel.setAiDetection(new ExportAiDetection(endpoint, "POST", null))`. The Python and JavaScript SDKs have no setter for it, so set the field on the export model before you call export: `export.aiDetection = {"endpoint": endpoint, "verb": "POST"}` in Python, or `exportModel.aiDetection = { endpoint, verb: 'POST' }` in JavaScript. Both SDKs send it with the rest of the model.

    The uploaded file is the [AI Text Detection Result](/reference/data-types/authenticity/results/ai-detection). Its positions point into the same crawled version `text.value` and also count UTF-16 code units:
    - `results` splits the text into sections. A `classification` of `2` marks text flagged as AI-generated, and `1` marks human-written text. Each section's `matches[].text.chars` holds its `starts` and `lengths`.
    - `summary.ai` is the AI score: the share of the text's words flagged as AI-generated, from `0` to `1`. It is not a probability.

    Collect the flagged ranges and pass them to the same `toSegments` function, as their own layer:

    <CodeGroup>
      ```javascript title="JavaScript" icon="square-js"
      function collectAiRanges(aiDetection) {
        const ranges = [];
        for (const section of aiDetection.results) {
          if (section.classification !== 2) continue; // 2 = AI-generated, 1 = human-written
          for (const match of section.matches) {
            const { starts, lengths } = match.text.chars;
            starts.forEach((start, i) => ranges.push({ start, end: start + lengths[i], type: 'ai' }));
          }
        }
        return ranges;
      }
      ```
      ```python title="Python" icon="python"
      def collect_ai_ranges(ai_detection):
          ranges = []
          for section in ai_detection["results"]:
              if section["classification"] != 2:  # 2 = AI-generated, 1 = human-written
                  continue
              for match in section["matches"]:
                  chars = match["text"]["chars"]
                  for start, length in zip(chars["starts"], chars["lengths"]):
                      ranges.append((start, start + length, "ai"))
          return ranges
      ```
      ```java title="Java" icon="java"
      // Add to ReportHighlighter, and import com.google.gson.JsonElement
      static List<Range> collectAiRanges(JsonObject aiDetection) {
          List<Range> ranges = new ArrayList<>();
          for (JsonElement element : aiDetection.getAsJsonArray("results")) {
              JsonObject section = element.getAsJsonObject();
              if (section.get("classification").getAsInt() != 2) continue; // 2 = AI-generated, 1 = human-written
              for (JsonElement match : section.getAsJsonArray("matches")) {
                  JsonObject chars = match.getAsJsonObject().getAsJsonObject("text").getAsJsonObject("chars");
                  JsonArray starts = chars.getAsJsonArray("starts");
                  JsonArray lengths = chars.getAsJsonArray("lengths");
                  for (int i = 0; i < starts.size(); i++) {
                      int start = starts.get(i).getAsInt();
                      ranges.add(new Range(start, start + lengths.get(i).getAsInt(), "ai"));
                  }
              }
          }
          return ranges;
      }
      ```
    </CodeGroup>
  </Step>

  <Step title="Add the PDF report (optional)">
    If the scan was submitted with `properties.pdf.create: true`, you can offer the PDF report as a download next to your own report. Add a `pdfReport` item to the export request:

    ```json
    "pdfReport": {
      "endpoint": "https://your-server.com/webhook/export/my-export-1/pdf-report",
      "verb": "POST"
    }
    ```

    The file is uploaded with the `application/pdf` content type. An export that asks for `pdfReport` or `aiDetection` without the matching submit property is rejected with `400 Bad Request`.
  </Step>

  <Step title="Summary">
    You now have everything a custom report needs: the list of sources and the totals from the completed webhook, the document text from the crawled version, and the match positions from each result. Your report shows reviewers where the matches and flagged passages are, so they can examine the evidence themselves.
  </Step>
</Steps>

## Next steps

<CardGroup cols={2}>
  <Card title="Export" icon="file-export" href="/reference/actions/downloads/export">See every artifact you can export and its options.</Card>
  <Card title="New Plagiarism Result" icon="magnifying-glass" href="/reference/data-types/authenticity/results/new-plagiarism-result">Review the full structure of an exported result.</Card>
  <Card title="Crawled Version" icon="file" href="/reference/data-types/authenticity/results/crawled-version">Review the full structure of the scanned document's text.</Card>
  <Card title="Install Open-Source Web Report" icon="code-branch" href="/guides/display/install-open-source-report">Prefer a ready-made interface you can theme? Use the Open-Source Web Report.</Card>
</CardGroup>

## Support

Should you require any assistance or have inquiries, contact [**Copyleaks Support**](https://help.copyleaks.com/hc/en-us/requests/new) or ask a question on [**Stack Overflow**](https://stackoverflow.com/questions/tagged/copyleaks-api) with the `copyleaks-api` tag.