How to load data from Weatherstack to Databricks Lakehouse

Begin by obtaining an API key from Weatherstack. Sign up on their website if you haven't already, and navigate to the API section to find your key. This key will be required to authenticate your requests to the Weatherstack API.

Use Python's `requests` library to fetch data from Weatherstack. Construct your API request URL with appropriate parameters (e.g., location, data type). Ensure your API key is included in the request. For example:
```python
import requests

api_key = 'YOUR_WEATHERSTACK_API_KEY'
endpoint = f'http://api.weatherstack.com/current?access_key={api_key}&query=New York'
response = requests.get(endpoint)
data = response.json()
```

After retrieving the JSON data from Weatherstack, parse and transform it into a structured format suitable for storage. You can use Python's `pandas` library to convert the JSON data into a DataFrame:
```python
import pandas as pd

df = pd.json_normalize(data)
```

Once you have the data in a DataFrame, save it locally in a format compatible with Databricks, such as CSV or Parquet. For example, saving as a CSV:
```python
df.to_csv('weather_data.csv', index=False)
```

Use Databricks CLI or UI to upload your saved file to the Databricks File System. If using the CLI, ensure it is installed and authenticated. Then, execute the command:
```bash
databricks fs cp weather_data.csv dbfs:/FileStore/weather_data.csv
```

In Databricks, use a notebook or a SQL query to load the CSV data from DBFS into a Databricks table. For instance, using PySpark:
```python
df_spark = spark.read.csv('dbfs:/FileStore/weather_data.csv', header=True, inferSchema=True)
df_spark.createOrReplaceTempView('weather_data_temp')

spark.sql('CREATE TABLE weather_data AS SELECT * FROM weather_data_temp')
```

Finally, verify the data has been loaded correctly by querying the Databricks table. Check for data consistency and completeness to ensure successful migration. Use a simple SQL query:
```sql
SELECT * FROM weather_data LIMIT 10;
```

This step-by-step guide outlines the process to manually move data from Weatherstack to Databricks Lakehouse without using third-party connectors or integrations.