# Some Pandas Techniques

Pandas is a python library used for data analysis, it is built on top of matplotlib and numpy for data visualization and mathematical operations respectively.

In this blog we would like to explore some features of pandas library, so we will use telecom churn data set as data source, and apply some of the desired techniques.

1. we import the required libraries, numpy and pandas.

```
import numpy as np
import pandas as pd
```

2. we read the data using read_csv from pandas library:

`data = pd.read_csv("telecom_churn.csv", on_bad_lines='skip')`

This is loading, it works as if we are converting csv data into pandas dataframe for pandas to be able to deal with it.

while a dataframe is like a spreadsheet or a table.

3. Lets take a quick into data using head():

`data.head()`

4. we can look into data shape using: data.shape which results in: 3333 rows X 20 columns, this is the shape of the entire data set.

5. The same way we can print out the names of all columns using :

```
data.columns
Index(['State', 'Account length', 'Area code', 'International plan', 'Voice mail plan', 'Number vmail messages', 'Total day minutes', 'Total day calls', 'Total day charge', 'Total eve minutes', 'Total eve calls', 'Total eve charge', 'Total night minutes', 'Total night calls', 'Total night charge', 'Total intl minutes', 'Total intl calls', 'Total intl charge', 'Customer service calls', 'Churn'], dtype='object')
```

6. we still also can take a look into the data using data.info() to list the types of each column:

```
data.info()
<class 'pandas.core.frame.DataFrame'>
RangeIndex: 3333 entries, 0 to 3332
Data columns (total 20 columns):
# Column Non-Null Count Dtype
--- ------ -------------- -----
0 State 3333 non-null object
1 Account length 3333 non-null int64
2 Area code 3333 non-null int64
3 International plan 3333 non-null object
4 Voice mail plan 3333 non-null object
5 Number vmail messages 3333 non-null int64
6 Total day minutes 3333 non-null float64
7 Total day calls 3333 non-null int64
8 Total day charge 3333 non-null float64
9 Total eve minutes 3333 non-null float64
10 Total eve calls 3333 non-null int64
11 Total eve charge 3333 non-null float64
12 Total night minutes 3333 non-null float64
13 Total night calls 3333 non-null int64
14 Total night charge 3333 non-null float64
15 Total intl minutes 3333 non-null float64
16 Total intl calls 3333 non-null int64
17 Total intl charge 3333 non-null float64
18 Customer service calls 3333 non-null int64
19 Churn 3333 non-null bool
dtypes: bool(1), float64(8), int64(8), object(3) memory usage: 498.1+ KB
```

7. Another technique is converting columns type from one to another, such as converting churn from boolean to int64:

`data["Churn"] = data["Churn"].astype("int64")`

8. beside converting types, another primary technique is using describe() to apply primary statistical computations on the columns of the numerical types, such as displaying mean, standard deviation, counts, quartiles, and maximum and minimum.

`data.describe() `

9. In this data set, we are mainly exploring users churns who are loyal to the company or not (Yes or NO) and the factors impacting their decision. One of the useful techniques is exploring the distribution of the Yes, No users of this telecommunication company , using value_counts method:

```
data["Churn"].value_counts()
0 2850
1 483
Name: Churn, dtype: int64
```

we can normalized these values to display fractions:

`data["Churn"].value_counts(normalize=True)`

10. sorting the dataframe according to one value: column is usually a an important feature in pandas, it an be done as the following:

`data.sort_values(by="Total day minutes", ascending=False).head()`

or sorting according to multiple columns:

`data.sort_values(by=["Churn", "Total day charge"], ascending=[True, False]).head()`

11 To inspect more we can add a column to this data set, which represents the total value of all telephone usage daily by adding four columns together as the following:

`data['Total calls'] = data['Total day calls'] + data['Total eve calls'] + data['Total night calls'] + data['Total intl calls']`

12. Another technique we might use for aggregating values into one table is pivot table, pivot tables make it easy to apply one statistical computation on multiple columns and view it according to one another column, as the following:

```
data.pivot_table(
["Total day calls", "Total eve calls", "Total night calls"],
["Area code"],
aggfunc="mean",
)
-------------------------------------------------------------
Total day calls Total eve calls Total night calls
Area code
408 100.50 99.79 99.04
415 100.58 100.50 100.40
510 100.10 99.67 100.60
```

The last technique we want to discuss here is time series using pandas:

we can use pandas for time series analysis as the following:

```
import pandas as pd
from datetime import datetime
import numpy as np
range_date = pd.date_range(start ='1/1/2020', end ='1/05/2020', freq ='Min')
print(range_date)
```

here we created a timestamp by minutes (freq='Min') starting from 01/01/2020 until 01/05/2020, and this was the result:

```
DatetimeIndex(['2020-01-01 00:00:00', '2020-01-01 00:01:00', '2020-01-01 00:02:00', '2020-01-01 00:03:00', '2020-01-01 00:04:00', '2020-01-01 00:05:00', '2020-01-01 00:06:00', '2020-01-01 00:07:00', '2020-01-01 00:08:00', '2020-01-01 00:09:00', ... '2020-01-04 23:51:00', '2020-01-04 23:52:00', '2020-01-04 23:53:00', '2020-01-04 23:54:00', '2020-01-04 23:55:00', '2020-01-04 23:56:00', '2020-01-04 23:57:00', '2020-01-04 23:58:00', '2020-01-04 23:59:00', '2020-01-05 00:00:00'], dtype='datetime64[ns]', length=5761, freq='T')
```

The length of the datetime stamp is 5761.the data type as datetime64[ns]. Pandas uses this type.

```
df = pd.DataFrame(range_date, columns =['date'])
df['data'] = np.random.randint(0, 100, size =(len(range_date)))
print(df.head(10))
```

Now, we are converting this time series into dataframe using the random function to generate random data.

```
date data
0 2020-01-01 00:00:00 86
1 2020-01-01 00:01:00 65
2 2020-01-01 00:02:00 15
3 2020-01-01 00:03:00 17
4 2020-01-01 00:04:00 12
5 2020-01-01 00:05:00 10
6 2020-01-01 00:06:00 16
7 2020-01-01 00:07:00 22
8 2020-01-01 00:08:00 54
9 2020-01-01 00:09:00 40
```

This was a quick look at some of pandas techniques in manipulating different types of data: category, integers, time series.

Thanks for reading so far, if you like this article please follow me on twitter on: @sanaomaro.

Resources:

Kashnitsky. (2021, May 7). Topic 1. exploratory data analysis with Pandas. Kaggle. Retrieved February 28, 2022, from https://www.kaggle.com/kashnitsky/topic-1-exploratory-data-analysis-with-pandas/notebook

Pandas: Python library - mode. Mode Resources. (2016, May 23). Retrieved February 28, 2022, from https://mode.com/python-tutorial/libraries/pandas/

Pandas: Basic of time series manipulation. GeeksforGeeks. (2021, August 31). Retrieved February 28, 2022, from https://www.geeksforgeeks.org/pandas-basic-of-time-series-manipulation/

## Comments