Python JSON

Learn how to work with JSON data in Python using the json module for parsing and generating JSON.

JSON in Python

JSON is a syntax for storing and exchanging data.

JSON is text, written with JavaScript object notation.

Python has a built-in package called json, which can be used to work with JSON data.

Example - Import the json module:

import json

Parse JSON - Convert from JSON to Python

If you have a JSON string, you can parse it by using the json.loads() method.

The result will be a Python dictionary.

Example - Convert from JSON to Python:

import json

# some JSON:
x = '{ "name":"John", "age":30, "city":"New York"}'

# parse x:
y = json.loads(x)

# the result is a Python dictionary:
print(y["age"])

Convert from Python to JSON

If you have a Python object, you can convert it into a JSON string by using the json.dumps() method.

Example - Convert from Python to JSON:

import json

# a Python object (dict):
x = {
  "name": "John",
  "age": 30,
  "city": "New York"
}

# convert into JSON:
y = json.dumps(x)

# the result is a JSON string:
print(y)

Convert Python objects into JSON strings

You can convert Python objects of the following types, into JSON strings:

dict
list
tuple
string
int
float
True
False
None

Example - Convert Python objects into JSON strings:

import json

print(json.dumps({"name": "John", "age": 30}))
print(json.dumps(["apple", "bananas"]))
print(json.dumps(("apple", "bananas")))
print(json.dumps("hello"))
print(json.dumps(42))
print(json.dumps(31.76))
print(json.dumps(True))
print(json.dumps(False))
print(json.dumps(None))

Python to JSON Conversion Table

When you convert from Python to JSON, Python objects are converted into the JSON (JavaScript) equivalent:

dict

Object

list

Array

tuple

Array

str

String

int

Number

float

Number

True

true

False

false

None

null

Format the Result

The example above prints a JSON string, but it is not very easy to read, with no indentations and line breaks.

The json.dumps() method has parameters to make it easier to read the result:

Example - Use the indent parameter to define the numbers of indents:

import json

x = {
  "name": "John",
  "age": 30,
  "married": True,
  "divorced": False,
  "children": ("Ann","Peter","James"),
  "pets": None,
  "cars": [
    {"model": "BMW 230", "mpg": 27.5},
    {"model": "Ford Edge", "mpg": 24.1}
  ]
}

print(json.dumps(x, indent=4))

You can also define the separators, default value is (", ", ": "), which means using a comma and a space to separate each object, and a colon and a space to separate keys from values:

Example - Use the separators parameter to change the default separator:

import json

x = {
  "name": "John",
  "age": 30,
  "married": True,
  "divorced": False,
  "children": ("Ann","Peter","James"),
  "pets": None,
  "cars": [
    {"model": "BMW 230", "mpg": 27.5},
    {"model": "Ford Edge", "mpg": 24.1}
  ]
}

print(json.dumps(x, indent=4, separators=(". ", " = ")))

Order the Result

The json.dumps() method has parameters to order the keys in the result:

Example - Use the sort_keys parameter to specify if the result should be sorted or not:

import json

x = {
  "name": "John",
  "age": 30,
  "married": True,
  "divorced": False,
  "children": ("Ann","Peter","James"),
  "pets": None,
  "cars": [
    {"model": "BMW 230", "mpg": 27.5},
    {"model": "Ford Edge", "mpg": 24.1}
  ]
}

print(json.dumps(x, indent=4, sort_keys=True))

Working with JSON Files

You can read and write JSON data to files using the json module:

Example - Writing JSON to a file:

import json

data = {
    "employees": [
        {"name": "John", "age": 30, "department": "IT"},
        {"name": "Jane", "age": 25, "department": "HR"},
        {"name": "Bob", "age": 35, "department": "Finance"}
    ],
    "company": "Tech Corp",
    "founded": 2010
}

# Write JSON to file
with open('data.json', 'w') as file:
    json.dump(data, file, indent=4)

print("Data written to data.json")

Example - Reading JSON from a file:

import json

# Read JSON from file
try:
    with open('data.json', 'r') as file:
        data = json.load(file)
    
    print("Company:", data['company'])
    print("Founded:", data['founded'])
    print("\nEmployees:")
    for employee in data['employees']:
        print(f"- {employee['name']}, Age: {employee['age']}, Dept: {employee['department']}")
        
except FileNotFoundError:
    print("File not found!")
except json.JSONDecodeError:
    print("Invalid JSON format!")

Handling JSON Errors

When working with JSON, you should handle potential errors:

Example - Error handling with JSON:

import json

def safe_json_parse(json_string):
    """Safely parse JSON string with error handling."""
    try:
        return json.loads(json_string)
    except json.JSONDecodeError as e:
        print(f"JSON decode error: {e}")
        return None
    except TypeError as e:
        print(f"Type error: {e}")
        return None

# Valid JSON
valid_json = '{"name": "John", "age": 30}'
result = safe_json_parse(valid_json)
print("Valid JSON result:", result)

# Invalid JSON
invalid_json = '{"name": "John", "age": 30,}'  # Extra comma
result = safe_json_parse(invalid_json)
print("Invalid JSON result:", result)

# Not a string
not_string = {"name": "John"}
result = safe_json_parse(not_string)
print("Not string result:", result)

Custom JSON Encoding

You can create custom JSON encoders for complex objects:

Example - Custom JSON encoder:

import json
from datetime import datetime, date

class DateTimeEncoder(json.JSONEncoder):
    """Custom JSON encoder for datetime objects."""
    
    def default(self, obj):
        if isinstance(obj, datetime):
            return obj.isoformat()
        elif isinstance(obj, date):
            return obj.isoformat()
        elif hasattr(obj, '__dict__'):
            return obj.__dict__
        return super().default(obj)

class Person:
    def __init__(self, name, age, birth_date):
        self.name = name
        self.age = age
        self.birth_date = birth_date
        self.created_at = datetime.now()

# Create a person object
person = Person("John Doe", 30, date(1993, 5, 15))

# Convert to JSON using custom encoder
json_string = json.dumps(person, cls=DateTimeEncoder, indent=2)
print("Custom encoded JSON:")
print(json_string)

# Parse it back
parsed_data = json.loads(json_string)
print("\nParsed data:")
print(parsed_data)

Working with Nested JSON

Handle complex nested JSON structures:

Example - Nested JSON operations:

import json

# Complex nested JSON
nested_data = {
    "users": [
        {
            "id": 1,
            "name": "John Doe",
            "contact": {
                "email": "john@example.com",
                "phone": "+1-555-0123",
                "address": {
                    "street": "123 Main St",
                    "city": "New York",
                    "country": "USA"
                }
            },
            "preferences": {
                "theme": "dark",
                "notifications": True,
                "languages": ["en", "es"]
            }
        },
        {
            "id": 2,
            "name": "Jane Smith",
            "contact": {
                "email": "jane@example.com",
                "phone": "+1-555-0456",
                "address": {
                    "street": "456 Oak Ave",
                    "city": "Los Angeles",
                    "country": "USA"
                }
            },
            "preferences": {
                "theme": "light",
                "notifications": False,
                "languages": ["en"]
            }
        }
    ],
    "metadata": {
        "total_users": 2,
        "last_updated": "2023-12-07T10:30:00Z"
    }
}

def find_user_by_id(data, user_id):
    """Find a user by ID in nested JSON."""
    for user in data.get("users", []):
        if user.get("id") == user_id:
            return user
    return None

def get_user_city(data, user_id):
    """Get user's city from nested structure."""
    user = find_user_by_id(data, user_id)
    if user:
        return user.get("contact", {}).get("address", {}).get("city")
    return None

def update_user_preference(data, user_id, preference_key, value):
    """Update a user's preference."""
    user = find_user_by_id(data, user_id)
    if user:
        user.setdefault("preferences", {})[preference_key] = value
        return True
    return False

# Usage examples
print("User 1 city:", get_user_city(nested_data, 1))
print("User 2 city:", get_user_city(nested_data, 2))

# Update preference
update_user_preference(nested_data, 1, "theme", "light")
print("Updated user 1 theme:", nested_data["users"][0]["preferences"]["theme"])

# Pretty print the entire structure
print("\nComplete data structure:")
print(json.dumps(nested_data, indent=2))

JSON Schema Validation

Validate JSON data against a schema (requires jsonschema package):

Example - JSON schema validation:

# Note: This requires 'pip install jsonschema'
# For demonstration purposes only

import json

# Define a simple schema
user_schema = {
    "type": "object",
    "properties": {
        "name": {"type": "string"},
        "age": {"type": "number", "minimum": 0},
        "email": {"type": "string", "format": "email"}
    },
    "required": ["name", "age"]
}

def validate_user_data(data):
    """Simple validation without external library."""
    errors = []
    
    if not isinstance(data, dict):
        errors.append("Data must be an object")
        return errors
    
    # Check required fields
    if "name" not in data:
        errors.append("Missing required field: name")
    elif not isinstance(data["name"], str):
        errors.append("Name must be a string")
    
    if "age" not in data:
        errors.append("Missing required field: age")
    elif not isinstance(data["age"], (int, float)) or data["age"] < 0:
        errors.append("Age must be a non-negative number")
    
    # Check optional email
    if "email" in data and not isinstance(data["email"], str):
        errors.append("Email must be a string")
    
    return errors

# Test data
valid_user = {"name": "John Doe", "age": 30, "email": "john@example.com"}
invalid_user = {"name": 123, "age": -5}

print("Validating valid user:")
errors = validate_user_data(valid_user)
if errors:
    print("Errors:", errors)
else:
    print("Valid!")

print("\nValidating invalid user:")
errors = validate_user_data(invalid_user)
if errors:
    print("Errors:", errors)
else:
    print("Valid!")

Practical JSON Examples

Configuration File Manager

import json
import os

class ConfigManager:
    def __init__(self, config_file="config.json"):
        self.config_file = config_file
        self.config = self.load_config()
    
    def load_config(self):
        """Load configuration from file."""
        if os.path.exists(self.config_file):
            try:
                with open(self.config_file, 'r') as file:
                    return json.load(file)
            except (json.JSONDecodeError, IOError) as e:
                print(f"Error loading config: {e}")
                return self.get_default_config()
        else:
            return self.get_default_config()
    
    def get_default_config(self):
        """Return default configuration."""
        return {
            "database": {
                "host": "localhost",
                "port": 5432,
                "name": "myapp"
            },
            "logging": {
                "level": "INFO",
                "file": "app.log"
            },
            "features": {
                "debug_mode": False,
                "cache_enabled": True
            }
        }
    
    def save_config(self):
        """Save configuration to file."""
        try:
            with open(self.config_file, 'w') as file:
                json.dump(self.config, file, indent=4)
            return True
        except IOError as e:
            print(f"Error saving config: {e}")
            return False
    
    def get(self, key_path, default=None):
        """Get configuration value using dot notation."""
        keys = key_path.split('.')
        value = self.config
        
        for key in keys:
            if isinstance(value, dict) and key in value:
                value = value[key]
            else:
                return default
        
        return value
    
    def set(self, key_path, value):
        """Set configuration value using dot notation."""
        keys = key_path.split('.')
        config = self.config
        
        for key in keys[:-1]:
            if key not in config:
                config[key] = {}
            config = config[key]
        
        config[keys[-1]] = value

# Usage example
config = ConfigManager()

print("Database host:", config.get("database.host"))
print("Debug mode:", config.get("features.debug_mode"))

# Update configuration
config.set("features.debug_mode", True)
config.set("database.port", 3306)

# Save changes
if config.save_config():
    print("Configuration saved successfully!")

print("Updated config:")
print(json.dumps(config.config, indent=2))

API Response Handler

import json
from urllib.request import urlopen
from urllib.error import URLError

class APIClient:
    def __init__(self, base_url):
        self.base_url = base_url
    
    def fetch_data(self, endpoint):
        """Fetch JSON data from API endpoint."""
        url = f"{self.base_url}/{endpoint}"
        
        try:
            with urlopen(url) as response:
                if response.status == 200:
                    data = response.read().decode('utf-8')
                    return json.loads(data)
                else:
                    print(f"HTTP Error: {response.status}")
                    return None
        except URLError as e:
            print(f"URL Error: {e}")
            return None
        except json.JSONDecodeError as e:
            print(f"JSON Decode Error: {e}")
            return None
    
    def process_user_data(self, user_data):
        """Process user data from API response."""
        if not user_data:
            return None
        
        processed = {
            "id": user_data.get("id"),
            "name": user_data.get("name", "Unknown"),
            "email": user_data.get("email", "No email"),
            "address": self.format_address(user_data.get("address", {})),
            "company": user_data.get("company", {}).get("name", "No company")
        }
        
        return processed
    
    def format_address(self, address):
        """Format address from API data."""
        if not address:
            return "No address"
        
        parts = [
            address.get("street", ""),
            address.get("city", ""),
            address.get("zipcode", "")
        ]
        
        return ", ".join(filter(None, parts))

# Example usage (would work with a real API)
# api = APIClient("https://jsonplaceholder.typicode.com")
# user_data = api.fetch_data("users/1")
# 
# if user_data:
#     processed = api.process_user_data(user_data)
#     print("Processed user data:")
#     print(json.dumps(processed, indent=2))

# Simulated API response for demonstration
simulated_response = {
    "id": 1,
    "name": "Leanne Graham",
    "email": "Sincere@april.biz",
    "address": {
        "street": "Kulas Light",
        "city": "Gwenborough",
        "zipcode": "92998-3874"
    },
    "company": {
        "name": "Romaguera-Crona"
    }
}

api = APIClient("https://example.com")
processed = api.process_user_data(simulated_response)
print("Processed user data:")
print(json.dumps(processed, indent=2))