How to load data from SFTP to MongoDB

Begin by ensuring that you have the necessary tools installed on your system. This includes Python for scripting, along with the `pymongo` library to interact with MongoDB and the `paramiko` library to access SFTP servers. You can install these using pip:
```bash
pip install pymongo paramiko
```

Use the `paramiko` library to establish a connection to the SFTP server. You will need the hostname, username, password, and the port (usually 22) of the SFTP server.
```python
import paramiko

transport = paramiko.Transport(('sftp.example.com', 22))
transport.connect(username='your_username', password='your_password')

sftp = paramiko.SFTPClient.from_transport(transport)
```

Navigate to the directory on the SFTP server where your data files are stored. Use the SFTP client to list files and download them to a local directory.
```python
files = sftp.listdir('/remote/directory')
for file in files:
sftp.get(f'/remote/directory/{file}', f'/local/directory/{file}')
```

Once the data files are downloaded, process them to a format suitable for MongoDB. This might involve reading CSV or JSON files and converting them into dictionaries or lists of dictionaries.
```python
import json

with open('/local/directory/data.json', 'r') as file:
data = json.load(file) # Assuming JSON format
```

Establish a connection to your MongoDB database using `pymongo`. You'll need the MongoDB URI, which typically includes the hostname, database name, and authentication details.
```python
from pymongo import MongoClient

client = MongoClient('mongodb://your_mongo_host:27017/')
db = client['your_database']
collection = db['your_collection']
```

Use the collection object to insert the processed data into MongoDB. Depending on your needs, you might use `insert_one()` for single documents or `insert_many()` for multiple documents.
```python
if isinstance(data, list):
collection.insert_many(data)
else:
collection.insert_one(data)
```

After insertion, verify that the data has been correctly stored in MongoDB by querying the collection. Once confirmed, close all connections and clean up any temporary files.
```python
for doc in collection.find():
print(doc)

sftp.close()
transport.close()
client.close()
```
Clean up local files if no longer needed:
```python
import os

for file in files:
os.remove(f'/local/directory/{file}')
```

By following these steps, you can efficiently transfer data from an SFTP server to a MongoDB database using Python without relying on third-party integrations.