How to use JSON to transfer data and bypass IP bans with proxies?
Everyone who develops applications, parses or works with APIs exchanges data between servers.
The JSON (JavaScript Object Notation) format is one of the most convenient formats for this. It's a simple way to package data into text that is easy for humans and machines to read. That's why it's supported by most systems and used by developers, parsing specialists and other pros.
In this article, we'll tell you about it:
- How proxy servers help bypass IP blocking and provide anonymity.
- How to use JSON in JavaScript.
- How proxies from Proxys.io can improve your data experience.
This will be of interest to both developers and parsing specialists.
Bans for exceeding the limit on requests to the server
Experts know that if you frequently send a request to the server and exceed the limit, you will be banned by IP. That's why parsing specialists and developers use proxy rotation and thus add more IPs.
When there are more IPs - the load on the server is distributed among them and so you are below the limit. IP will not be banned and you will finish the same parsing without delays.
The simplicity and convenience of JSON
As we wrote in the introduction: basically, JSON is a simple way to package data in a text format. Here are the main characteristics of JSON:
- Objects. They are a set of “key: value” pairs enclosed in curly brackets {}.
- Arrays. Represent an ordered list of values enclosed in square brackets [].
- Keys. Strings that serve to name values. They must always be enclosed in double quotes.
- Values. Can be strings, numbers, objects, arrays, boolean values (true or false), or null.
JSON Example:
{
"name": "John",
"age": 30,
"isStudent": false,
"courses": ["Math", "Science"],
"address": {
"street": "123 Main St",
"city": "New York"
}
}
In this example:
- “name”, ‘age’, ‘isStudent’, ‘courses’, and ‘address’ are keys.
- “John“, 30, false, [”Math“, ‘Science’], and {”street": “123 Main St”, ‘city’: ‘New York’} are values.
This format is used to transfer data from server to client or to store data after parsing. However, if the server bans the IP, you will not be able to access the data.
How do proxies help avoid IP bans?
A parsing specialist knows about the risk of IP bans due to a large number of requests. That's why he rents a pool of proxies and sets up rotation. This way he will avoid not only bans, but also will not have to enter captchas. Because, captchas are given to pass also after exceeding the limit.
Here's how it works:
- The specialist rents several proxies. They can be proxies from the same country and from different regions, or from different countries.
- He configures IP rotation so that requests are distributed among all rented proxies. This means that each request is sent through a new IP from the list.
- With the help of rotation, the specialist continues parsing and does not exceed the limit on requests to the server.
Thus, renting and rotating multiple proxies allows you to complete parsing quickly and without problems.
Examples of JSON and proxy usage
Fetch API allows you to download data, including JSON files. If you make frequent requests, your IP may be banned. Proxies from Proxys.io will help bypass this risk.
Example of using Fetch API with a proxy:
fetch('https://scraping-demo-api-json-server.vercel.app/products', {
method: 'GET',
headers: {
'Proxy-Authorization': 'Basic YOUR_PROXY_AUTH',
},
mode: 'cors',
})
.then(response => response.json())
.then(data => console.log(data))
.catch(error => console.error('Error during data retrieval:', error));
Working with local JSON files
If you need to work with local files, use the FileReader API. Here's how to do it:
<input type="file" id="fileInput" accept=".json" />
<div id="productList"></div>
<script>
document.getElementById('fileInput').addEventListener('change', function (event) {
const file = event.target.files[0];
if (file) {
const reader = new FileReader();
reader.onload = function (e) {
const data = JSON.parse(e.target.result);
displayProducts(data.products);
};
reader.readAsText(file);
}
});
function displayProducts(products) {
products.forEach(product => {
console.log(`Product: ${product.name}, Цена: $${product.price}`);
});
}
</script>
Proxies can help bypass IP bans and improve JSON handling.