Reading and Writing JSON Files with Python JSON Module

With Python, you can read from and write data to a variety of data sources including CSV, TSV, JSON and databases. Several libraries are available in Python for data reading and writing tasks, including the CSV module and the sqlite3 module for databases. In this tutorial, we’ll show you how to read and write JSON files with Python using Python’s json module. At the end, we’ll explain how the pandas library makes it easier to read JSON files into tabular format.

What is a JSON File

JSON stands for JavaScript Object Notation which is one of the most widely used formats for data transfer and serialized information exchange between client server applications. JSON data is human-readable which makes it one of the most popular formats for transferring data over REST APIs.

Writing JSON Files with Python

JSON files store data in a specific format. One way to make JSON compliant data in Python is to create nested dictionaries where you have an outer dictionary containing multiple items in the form of key-value pairs. The key for each pair is a data attribute. The corresponding value for a key is another dictionary that contains records for the attribute. For instance, look at the department_dic in the following script.

departments_dic = {
         "Department_Name":{"0":"Development","1":"QA","2":"HR","3":"Finance", "4":"Configuration"},
         "Department_Manager":{"0":"Sally","1":"Joana","2":"Nick","3":"Robert", "5": "Sara"},
         "Department_Type":{"0":"Tech","1":"Tech","2":"Admin","3":"Admin", "4":"Tech"},
         "Total_Employees" :{"0":25,"1":15,"2":5,"3":5, "4":10}
}

Each item in the department_dic contains an attribute name as an item key, and a dictionary as an item value. For example, the first item of the department_dic, Department_Name, has 5 items: Development, QA, HR, Finance, Configuration. The index number is the key for each item in the nested dictionaries. For example, the index number for Development department is 0, and the index number for QA is 1. With nested dictionaries, each item in a dictionary depicts a column when represented in tabular format.

Another way to create a Python object that can be written to a JSON file is to initialize a list of dictionaries. Items within dictionaries correspond to a single record. The key for each item is the attribute name while the corresponding value represent the value for that attribute. For example, look at the department_list list in the following script. Here each dictionary in the list contains one complete record for a department including the Department_Name, Department_Manager, Department_Type, Total_Employees etc. Structured this way, each dictionary in the list corresponds to a row when represented in tabular format.

departments_list = [
    {"Department_Name":"Development", "Department_Manager":"Sally", "Department_Type":"Tech", "Total_Employees":25},
    {"Department_Name":"QA", "Department_Manager":"Joana", "Department_Type":"Tech", "Total_Employees":15},
    {"Department_Name":"HR", "Department_Manager":"Nick", "Department_Type":"Admin", "Total_Employees":5},
    {"Department_Name":"Finance", "Department_Manager":"Robert", "Department_Type":"Admin", "Total_Employees":5},
    {"Department_Name":"Configuration", "Department_Manager":"Sara", "Department_Type":"Tech", "Total_Employees":10}
]

We now have two objects that can be written to JSON files: department_dic and department_list. To write data to a JSON file, you can use the dump() function of the json module. The Python object that contains JSON data is passed to the dump()method. The following Python script writes two JSON files using department_dic and department_list objects.

import json
with open('G:/departments_dic.json', 'w') as jf:
    json.dump(departments_dic, jf)

with open('G:/departments_list.json', 'w') as jf:
    json.dump(departments_list, jf)

Get Our Python Developer Kit for Free

I put together a Python Developer Kit with over 100 pre-built Python scripts covering data structures, Pandas, NumPy, Seaborn, machine learning, file processing, web scraping and a whole lot more - and I want you to have it for free. Enter your email address below and I'll send a copy your way.

Reading JSON Files with Python JSON Module

Python’s json module can also be used to read JSON files. To do so, first you have to open the JSON file like any other file using the open() method. The object returned by the open() method is then passed to the load() method of the json module which returns data in JSON format. The following script reads the departments_dic.json file we created in the last section. The load() method returns a dictionary which is then iterated using a foreach loop and all the items of the dictionary are displayed on the console.

import json

json_file = open('G:/departments_dic.json',)


data = json.load(json_file)

for k,v in data.items():
    print (k,v)

Here is the output of the script above:

Output:

Department_Name {'0': 'Development', '1': 'QA', '2': 'HR', '3': 'Finance', '4': 'Configuration'}
Department_Manager {'0': 'Sally', '1': 'Joana', '2': 'Nick', '3': 'Robert', '5': 'Sara'}
Department_Type {'0': 'Tech', '1': 'Tech', '2': 'Admin', '3': 'Admin', '4': 'Tech'}
Total_Employees {'0': 25, '1': 15, '2': 5, '3': 5, '4': 10}

In the same way, the following script reads the departments_list.json file we created in the last section:

import json


json_file = open('G:/departments_list.json',)


data = json.load(json_file)

for i in data:
    print (i)

Output:

{'Department_Name': 'Development', 'Department_Manager': 'Sally', 'Department_Type': 'Tech', 'Total_Employees': 25}
{'Department_Name': 'QA', 'Department_Manager': 'Joana', 'Department_Type': 'Tech', 'Total_Employees': 15}
{'Department_Name': 'HR', 'Department_Manager': 'Nick', 'Department_Type': 'Admin', 'Total_Employees': 5}
{'Department_Name': 'Finance', 'Department_Manager': 'Robert', 'Department_Type': 'Admin', 'Total_Employees': 5}
{'Department_Name': 'Configuration', 'Department_Manager': 'Sara', 'Department_Type': 'Tech', 'Total_Employees': 10}

Reading JSON Files with Pandas

Though you can use the json module to read JSON files, you should use the read_json() function from the pandas module of Python if you want to read JSON files in tabular format. Here’s how you do it using our departments_dic.json file from earlier:

import pandas as pd
dep_dic = pd.read_json('G:/departments_dic.json')
dep_dic.head()

The read_json() function returns a pandas dataframe. The output below shows the JSON data in tabular format from our pandas dataframe.

Output:

In the same way, you can convert the departments_list.json file to a pandas dataframe using the script below:

import pandas as pd
dep_list = pd.read_json('G:/departments_list.json')
dep_list.head()

Output:

Converting Pandas Dataframe to JSON

Finally, you can convert a pandas dataframe to a JSON file using the to_json() method, as shown below.

import pandas as pd
dep_list = pd.read_json('G:/departments_list.json')
dep_list.to_json('G:/dep_list2.json', orient='records', lines=True)

If you open the dep_list2.json file you just created, you’ll see that it contains the following JSON data:

Output:

Get Our Python Developer Kit for Free

I put together a Python Developer Kit with over 100 pre-built Python scripts covering data structures, Pandas, NumPy, Seaborn, machine learning, file processing, web scraping and a whole lot more - and I want you to have it for free. Enter your email address below and I'll send a copy your way.