APIs

What is an API? 

API (Application Programming Interface) is a collection of tools that allows different applications to interact. 

In other words, an API is a set of function that allows application to access data and interact with external software components, operating systems, servers or micro-services.

To put it in a more simple way, an API delivers a users response to the system and send the system's response back to the user.

For example, you click on "Add to cart" in an Amazon website; an API tells the website you added an item to your cart and the website puts the item in your cart and your cart is updated.

Everyday Examples of API

• Searching for hotels
• A travel site may be linked to 10 other travel sites to find the best deals for you. When you input details like Victoria, 2 nights, 1 room, 2 adults, you send this request to those 10 different travel sites. The API takes your request with the inputs you have provided and pings the 10 sites, who in return,  send back the deals they have found. You then pick the best fit. Here the API is a messenger for your requests.

• Finding a Facebook profile
• Want to find your long lost childhood friend on Facebook? Just type in their name in the search box and the Facebook API informs Facebook's servers that you are looking for that specific name you have given. Facebook then gives you a list of all the profiles that match that name.

• Finding a new restaurant
• Let's say you are just bored of cooking and also in the mood of trying out something new. You grab your smartphone and look up restaurants near by. In a fraction of seconds, you are shown a dozen of restaurants near your home. Thanks to Google API; they are able to easily display all relevant information including ratings, business hours and even time they are likely to be busy.

• Staying up to date on Twitter
• You just want to know what's happening in the world of sports. You know that Twitter is the fastest news provider nowadays. So you open up Twitter and navigate to the 'Sports' section in your Twitter profile. Twitter's API allows you to easily see various tweets relating to Sports. You are even able to share your favorite team's score by retweeting the final score. From here Twitter knows how to take this tweet and display it to everyone who follows you.

Some of the biggest names on the web like Reddit, Spotify, Twitter and Facebook offer free APIs and make data available. Other services and companies charge for these APIs.

Using APIs, we can access web data, analyze the data, generate insights and make predictions.

API requests

APIs are hosted on web servers. API requests are essentially your program asking for data. The API usually returns this data in JSON format. We make an API request to the web servers with the data we want. The server then responds. In Python, API requests are made using the requests library. 

Requests is an elegant and simple HTTP library for Python, built for human beings.

GET request

There are many types of API requests. The most common is a GET request which we use for Data Collection.

Usage Example:

We can use a simple GET request to collect information from the OpenNotify API.

OpenNotify Endpoints:

http://api.open-notify.org/iss-now.json

http://api.open-notify.org/astros.json

http://api.open-notify.org/iss-pass.json?lat=LAT&lon=LON

An endpoint is a server route for retrieving specific data from an API.

iss-now.json endpoint gets the current latitude and longitude position of the ISS. 

Why not use a dataset for this task, you may ask? A dataset wouldn't be a good tool for this task because the information often changes and it involves some calculation on the server.

script.py

1

# Make a get request to get the latest position of the ISS from the OpenNotify API.

2

response = requests.get("http://api.open-notify.org/iss-now.json")

3

status_code = response.status_code

Variables

status_codeint (<class 'int'>)

200

responseResponse (<class 'mock_requests.Response'>)

<mock_requests.Response at 0x7f56498565c0>

Understanding Status Codes

Web servers returns status codes every time they receive an API request. A status code reports what happened with a request. Here are some of the codes that are relevant to GET requests:

• 200 - Everything went OK and the server returned a result (if any)
• 301 - The server is redirecting you to a different endpoint. This can happen when a company moves to a different domain or when an endpoint's name has changed.
• 401 - The server thinks you are not authenticated.
• 400 - The server thinks you made a bad request. 
• 403 - The resource you're trying to access is forbidden and you do not have the right permissions to see it.
• 404 - The server didn't find the resources you tried to access.

Examples:

script.py

1

#Status code 404

2

response = requests.get("http://api.open-notify.org/iss-pass")

3

status_code = response.status_code

4

​

5

#iss-pass wasn't a valid endpoint, so the API's server sent us a 404 status code in response. We forgot to add .json at the end, like the API documentation tells us to do.

Variables

responseResponse (<class 'mock_requests.Response'>)

<mock_requests.Response at 0x7f56497a1470>

status_codeint (<class 'int'>)

404

script.py

1

# Status code 400

2

response = requests.get("http://api.open-notify.org/iss-pass.json")

3

status_code = response.status_code

4

​

5

# we got a 400 status code, which indicates a bad request. If we look at the documentation for the OpenNotify API, we see that the ISS Pass endpoint requires two parameters.

Variables

responseResponse (<class 'mock_requests.Response'>)

<mock_requests.Response at 0x7f56497b0898>

status_codeint (<class 'int'>)

400

Adding query parameters as a dictionary

If we look at the documentation for the OpenNotify API, we see that the ISS Pass endpoint needs to parameters.

The ISS Pass endpoint tells us when the ISS will pass over a given location on the Earth. To request this information, we need to pass the coordinates for a specific location to the API. We do this by passing two parameters: latitude and longitude.

To do this, we can add an optional keyword argument, 'params', to our request. In this case, we need two parameters longitude and latitude. We can make a dictionary that contains these parameters and then pass them into the function.

We can also do the same thing directly by adding the query parameters to the url like this: http://api.open-notify.org/iss-pass.json?lat=40.71&lon=-74

However, it's always better to set up the parameters as a dictionary because the requests library takes care of the formatting.

script.py

1

# Set up the parameters we want to pass to the API.

2

# This is the latitude and longitude of New York City.

3

parameters = {"lat": 40.71, "lon": -74}

4

​

5

# Make a get request with the parameters.

6

response = requests.get("http://api.open-notify.org/iss-pass.json", params=parameters)

7

​

8

# Print the content of the response (the data the server returned)

9

print(response.content)

10

​

11

# This gets the same data as the command above

12

response = requests.get("http://api.open-notify.org/iss-pass.json?lat=40.71&lon=-74")

13

print(response.content)

14

​

15

#Set up the parameters we want to pass to the API 

16

#This is the latitude and longitude of SFO

17

parameters = {"lat": 37.78, "lon": -122.41}

18

​

19

#Make a get request with the parameters

20

response = requests.get("http://api.open-notify.org/iss-pass.json", params=parameters)

21

​

22

#Print the content of the response (the data the server returned)

23

content = response.content

24

print(content)

25

print(type(content))

Output

b'{\n "message": "success", \n "request": {\n "altitude": 100, \n "datetime": 1441417753, \n "latitude": 40.71, \n "longitude": -74.0, \n "passes": 5\n }, \n "response": [\n {\n "duration": 330, \n "risetime": 1441445639\n }, \n {\n "duration": 629, \n "risetime": 1441451226\n }, \n {\n "duration": 606, \n "risetime": 1441457027\n }, \n {\n "duration": 542, \n "risetime": 1441462894\n }, \n {\n "duration": 565, \n "risetime": 1441468731\n }\n ]\n}' b'{\n "message": "success", \n "request": {\n "altitude": 100, \n "datetime": 1441417753, \n "latitude": 40.71, \n "longitude": -74.0, \n "passes": 5\n }, \n "response": [\n {\n "duration": 329, \n "risetime": 1441445639\n }, \n {\n "duration": 629, \n "risetime": 1441451226\n }, \n {\n "duration": 606, \n "risetime": 1441457027\n }, \n {\n "duration": 542, \n "risetime": 1441462894\n }, \n {\n "duration": 565, \n "risetime": 1441468731\n }\n ]\n}' b'{\n "message": "success", \n "request": {\n "altitude": 100, \n "datetime": 1441417753, \n "latitude": 37.78, \n "longitude": -122.41, \n "passes": 5\n }, \n "response": [\n {\n "duration": 369, \n "risetime": 1441456672\n }, \n {\n "duration": 626, \n "risetime": 1441462284\n }, \n {\n "duration": 581, \n "risetime": 1441468104\n }, \n {\n "duration": 482, \n "risetime": 1441474000\n }, \n {\n "duration": 509, \n "risetime": 1441479853\n }\n ]\n}' <class 'bytes'>

JSON Format

API lets a developer make a specific "call" or "request" in order to send or receive data. This communication is done using a programming language called "JSON"(JavaScript Object Notation). 

In other words, JSON is the main format for sending and receiving data through APIs.

It can be used to make a defined action such as updating or deleting data.

There are four basic request methods that can be made with API:

They are:

• GET - Gathers data (Pulling all coupon codes)
• PUT - Updates pieces of data (Updating product pricing)
• POST - Creates (Creating a new product category)
• DELETE - Deletes (Deleting a blog post)

So what is JSON and why is it used?

JSON is used to represent data on a server. It is fairly easy to read by humans and easy for machines/applications to understand. It is outputted in key/value pairs. Keys are fixed objects defined by the application and will remain the same. The values will be associated to the key. For example, Subway is the Key and 24722 is the value.

Example:

{

    "Subway": 24722,

    "McDonalds": 14098,

    "Starbucks": 10821,

    "Pizza Hut": 7600

}

Python json library

Python offers great support for JSON through its json library. We can convert lists and dictionaries to JSON and JSON to lists and dictionaries. 

Our ISS Pass data, for example, is dictionary encoded as a string in JSON format.

{
  "message": "success", 
  "request": {
    "altitude": 100, 
    "datetime": 1626210005, 
    "latitude": 40.71, 
    "longitude": -74.0, 
    "passes": 5
  }, 
  "response": [
    {
      "duration": 220, 
      "risetime": 1626220537
    }, 
    {
      "duration": 632, 
      "risetime": 1626226088
    }, 
    {
      "duration": 638, 
      "risetime": 1626231894
    }, 
    {
      "duration": 569, 
      "risetime": 1626237784
    }, 
    {
      "duration": 585, 
      "risetime": 1626243649
    }
  ]
}

The Python JSON library has two main methods

• dumps - takes in a Python object and converts to a JSON string
• loads - takes in a JSON string and converts it to a Python object

script.py

1

# Make a list of fast food chains.

2

best_food_chains = ["Taco Bell", "Shake Shack", "Chipotle"]

3

print(best_food_chains)

4

print(type(best_food_chains))

5

​

6

# Import the JSON library.

7

import json

8

​

9

# Use json.dumps to convert best_food_chains to a string.

10

best_food_chains_string = json.dumps(best_food_chains)

11

print(best_food_chains_string)

12

print(type(best_food_chains_string))

13

​

14

# Convert best_food_chains_string back to a list.

15

print(json.loads(best_food_chains_string))

16

print(type(json.loads(best_food_chains_string)))

17

​

18

# Make a dictionary

19

fast_food_franchise = {

20

    "Subway": 24722,

21

    "McDonalds": 14098,

22

    "Starbucks": 10821,

23

    "Pizza Hut": 7600

24

}

25

​

26

# We can also dump a dictionary to a string and load it.

27

fast_food_franchise_string = json.dumps(fast_food_franchise)

28

print(fast_food_franchise_string)

29

print(type(fast_food_franchise_string))

30

​

31

fast_food_franchise_2 = json.loads(fast_food_franchise_string)

32

print(fast_food_franchise_2)

33

print(type(fast_food_franchise_2))

Output

['Taco Bell', 'Shake Shack', 'Chipotle'] <class 'list'> ["Taco Bell", "Shake Shack", "Chipotle"] <class 'str'> ['Taco Bell', 'Shake Shack', 'Chipotle'] <class 'list'> {"Subway": 24722, "Pizza Hut": 7600, "Starbucks": 10821, "McDonalds": 14098} <class 'str'> {'Subway': 24722, 'Pizza Hut': 7600, 'McDonalds': 14098, 'Starbucks': 10821} <class 'dict'>

Getting JSON from a request as Python Objects using .json()

We can get the content of the response as a Python object using the .json() method on the response.

script.py

1

# Set up parameters we want to pass to the API 

2

# Make a get request with the parameters

3

parameters = {"lat": 37.78, "lon": -122.41}

4

response = requests.get("http://api.open-notify.org/iss-pass.json", params=parameters)

5

​

6

# Get the response data as a Python object.  Verify that it's a dictionary.

7

json_data = response.json()

8

print(type(json_data))

9

print(json_data)

10

​

11

first_pass_duration = json_data["response"][0]["duration"]

12

print(first_pass_duration)

13

print(type(first_pass_duration))

Output

<class 'dict'> {'request': {'passes': 5, 'latitude': 37.78, 'longitude': -122.41, 'altitude': 100, 'datetime': 1441417753}, 'message': 'success', 'response': [{'duration': 369, 'risetime': 1441456672}, {'duration': 626, 'risetime': 1441462284}, {'duration': 581, 'risetime': 1441468104}, {'duration': 482, 'risetime': 1441474000}, {'duration': 509, 'risetime': 1441479853}]} 369 <class 'int'>

Response headers

When the server generates a response, it also sends metadata in addition to the status code and the requested data. The metadata has information on how it generated the data and how to decode it. This information appears in the response headers. We can access response headers using the .headers property.

The headers will appear as a dictionary as below.

script.py

1

# Headers is a dictionary

2

print(response.headers)

3

​

4

# Get content_type from response.headers

5

content_type = response.headers["content-type"]

6

print(content_type)

Output

{'content-length': '520', 'connection': 'keep-alive', 'date': 'Sat, 05 Sep 2015 01:49:13 GMT', 'content-type': 'application/json', 'via': '1.1 vegur', 'server': 'gunicorn/19.3.0'} application/json

Variables

content_typestr (<class 'str'>)

'application/json'

responseResponse (<class 'mock_requests.Response'>)

<mock_requests.Response at 0x7f50f6a5b240>

Find the number of people in space

script.py

1

# Call the API here.

2

response = requests.get("http://api.open-notify.org/astros.json")

3

​

4

# Get the response data as a Python object

5

json_data = response.json()

6

print(json_data)

7

​

8

in_space_count = json_data["number"]

9

print(in_space_count)

10

​

Output

{'people': [{'name': 'Gennady Padalka', 'craft': 'ISS'}, {'name': 'Mikhail Kornienko', 'craft': 'ISS'}, {'name': 'Scott Kelly', 'craft': 'ISS'}, {'name': 'Oleg Kononenko', 'craft': 'ISS'}, {'name': 'Kimiya Yui', 'craft': 'ISS'}, {'name': 'Kjell Lindgren', 'craft': 'ISS'}, {'name': 'Sergey Volkov', 'craft': 'ISS'}, {'name': 'Andreas Mogensen', 'craft': 'ISS'}, {'name': 'Aidyn Aimbetov', 'craft': 'ISS'}], 'number': 9, 'message': 'success'} 9

Here are the takeaways from this article

• What is an API?
• Everyday examples of API
• JSON format
• So what is JSON and why JSON is used?
• APIs requests (GET, PUT, POST, DELETE)
• Get request in detail
• Understanding status codes from API response
• Python JSON library
• Python JSON library methods
• Getting JSON from a request using .json()
• Response headers
• Final get request example - Finding the no.of people in space

Skills acquired:

• How to access web data using APIs (Data Collection using APIs)
• Querying and retrieving data using APIs
• How to create and process API requests for Data Collection (GET Requests)
• How to parse responses

Tools Used:

• Python's requests library
• Python's json library main methods - json.dumps() and json.loads()
• .json() method