Skip to main content

Learn about JSON data mapping

This guide provides a comprehensive overview of JSON data mapping, a fundamental process for modern system integration and data management. You will learn what JSON is, what data mapping entails, and how to apply these concepts to transform data effectively.

What is JSON?​

JSON (JavaScript Object Notation) is a lightweight, text-based format for data interchange. It is easy for humans to read and write and easy for machines to parse and generate. Due to its simplicity and flexibility, JSON has become the standard for data exchange between web servers and applications, especially for APIs.

JSON is built on two primary structures:

  • Objects: A collection of key/value pairs, enclosed in curly braces {}. Keys are strings, and values can be strings, numbers, booleans, arrays, or other objects.

  • Arrays: An ordered list of values, enclosed in square brackets [].

{
  "orderId": "12345",
  "customer": {
    "firstName": "John",
    "lastName": "Doe"
  },
  "items": [
    {
      "productId": "A-456",
      "quantity": 2
    },
    {
      "productId": "B-789",
      "quantity": 1
    }
  ],
  "isShipped": false
}

What is data mapping?​

Data mapping is the process of creating relationships between data fields from a source format to a target format. Think of it as creating a "map" that tells data from one system how to find its correct place in another system.

In the context of JSON, it means transforming an input JSON structure into a different output JSON structure. This is crucial when two systems need to communicate but use different data schemas.

Why is JSON data mapping essential?​

Data mapping is the engine behind seamless system integration. Its primary purposes are:

  • System Integration: When connecting two different applications (e.g., a CRM and an e-commerce platform), their APIs will almost certainly use different JSON structures. Mapping is required to translate the data so both systems can understand it.

  • Data Transformation: It’s not just about moving data. Mapping allows you to clean, enrich, restructure, and standardize data as it flows from one point to another. For example, you could combine a firstName and lastName from a source into a single fullName field in the target.

Core concepts & examples​

Let’s break down the process with a practical example. Imagine we have incoming orders from an e-commerce platform (source) and we need to send it to our internal fulfillment system (target).

Source JSON:​

[
  {
    "purchase_id": "ORD-9876",
    "client_details": {
      "firstname": "Jane",
      "lastname": "Smith",
      "contact_email": "jane.smith@example.com"
    },
    "line_items": [
      {
        "sku": "TS-BLUE-L",
        "item_name": "Blue T-Shirt - Large",
        "qty": 1,
        "price": 19.99
      },
      {
        "sku": "MUG-LOGO",
        "item_name": "Company Logo Mug",
        "qty": 2,
        "price": 9.5
      }
    ]
  },
  {
    "purchase_id": "ORD-9877",
    "client_details": {
      "firstname": "John",
      "lastname": "Doe",
      "contact_email": "john.doe@example.com"
    },
    "line_items": [
      {
        "sku": "TS-RED-M",
        "item_name": "Red T-Shirt - Medium",
        "qty": 2,
        "price": 19.99
      }
    ]
  }
]

Our fulfillment system requires the data in the following format:

Target JSON:​

[
  {
    "orderReference": "ORD-9876",
    "customerEmail": "jane.smith@example.com",
    "name": "Jane Smith",
    "products": [
      {
        "productId": "TS-BLUE-L",
        "quantity": 1
      },
      {
        "productId": "MUG-LOGO",
        "quantity": 2
      }
    ]
  },
  {
    "orderReference": "ORD-9877",
    "customerEmail": "john.doe@example.com",
    "name": "John Doe",
    "products": [
      {
        "productId": "TS-RED-M",
        "quantity": 1
      }
    ]
  }
]

Reader & writer​

Zparse natively supports both reading and writing JSON data, making it a powerful tool for any integration involving this popular format. These functions act as the entry and exit points for your data, allowing you to parse incoming JSON and construct new JSON to send to other systems.

JSON reader (parser)​

The JSON reader is responsible for interpreting and parsing incoming JSON data. It takes a raw JSON string or file and transforms it into a structured object that Zparse can understand and manipulate. This allows you to easily access every key and value within the JSON for use in your mapping.

A JSON reader node

Use Cases:

  • Processing API responses: When you receive a JSON response from an external API, the reader parses it so you can extract the specific data points you need.
  • Handling webhook payloads: Incoming webhooks from services like Stripe or GitHub send data in JSON format. The reader makes this data available to trigger and populate your workflows.
  • Reading uploaded files: If your process involves users uploading JSON files, the reader can parse the contents of those files for processing.

Example process:

A raw JSON string like {"customer": {"name": "John Doe"}} is fed into the reader. The reader then exposes the data, allowing you to access the customer’s name within your Zparse mapping.

JSON writer​

The JSON writer does the opposite of the reader. It constructs a new, well-formed JSON object or string from various data points within your Zparse workflow. You map your internal data to a target JSON structure, and the writer builds the final output.

A JSON writer node

Use Cases:

  • Creating API request bodies: When you need to send data to an external API, the writer assembles the required JSON payload from your workflow’s data.
  • Sending data to webhooks: Construct a custom JSON object to be sent as a payload to another application’s webhook.
  • Generating JSON file: Create structured JSON file for export or reporting.

Example Process:

You might have two separate variables in your workflow: order_id = "O-567" and status = "shipped". Using the writer, you can map these to a target structure to generate the final JSON output: {"orderReference":"O-567", "shippingStatus":"shipped"}.

Together, the JSON Reader and Writer provide a complete, end-to-end solution for handling JSON data, enabling seamless communication between Zparse and any external system.

Mapping rules & transformations​

To get from the source to the target, we apply a set of rules:

  1. Direct mapping (renaming): The field names are different, but the data is moved directly.
    • purchase_id → orderReference
    • client_details.contact_email → customerEmail

A graph which maps the order reference and the customer email

  1. Restructuring and mapping an array: We need to process the line_items array and create a new products array. For each item in the source array, we map its fields to the new structure.
    • line_items[].sku → products[].productId
    • line_items[].qty → products[].quantity
    • Fields like item_name and price from the source are ignored because they aren’t needed in the target.

A graph with a Map node to iterate on products details

A graph to collect products details

  1. Value transformation (concatenation): The name field does not exist in the source. We must concatenate the firstname and the lastname.
    • name = "Jane" + " " + "Smith" = "Jane Smith"

A graph to concatenate the firstname and the lastname of a customer