In Python, bytes are a built-in data type used to represent a sequence of bytes. They are immutable sequences of integers, with each integer typically representing a byte of data ranging from 0 to 255.

Convert Bytes Data into a Python Pandas Dataframe?

We can convert bytes into data frames using different methods:

1. Using the pd.DataFrame Constructor, bytes_data decoder and StringIO

We can convert bytes into data frames using pd.DataFrame constructor directly. Here, we created a byte data and stored it in a variable, we converted this byte data into string using the decode(‘utf-8’) method then the pd.read_csv method reads the string as a CSV and converts it into a DataFrame (df_method1).

import pandas as pd
from io import StringIO

# Sample bytes data
bytes_data = b'Name,Age,Occupation\nJohn,25,Engineer\nAlice,30,Doctor\nBob,28,Artist'

# Convert bytes to string and then to DataFrame
data_str = bytes_data.decode('utf-8')
df_method1 = pd.read_csv(StringIO(data_str))

# Display the DataFrame
print(df_method1)

Output:

    Name  Age Occupation
0   John   25   Engineer
1  Alice   30     Doctor
2    Bob   28     Artist

2. Using NumPy and io.BytesIO

The io.BytesIO class is part of Python’s built-in io module. It provides a way to create a file-like object that operates on in-memory bytes data.

Here, we use NumPy’s genfromtxt() function to read data from a CSV-like formatted byte stream. BytesIO(bytes_data) creates a file-like object that provides a stream interface to the bytes data. delimiter=’,’ names=True, dtype=None, and encoding=’utf-8′ specifies the parameters of the encoding.

Then we converted this array data into dataframe.

import numpy as np
import pandas as pd
from io import BytesIO

# Sample bytes data
bytes_data = b'Name,Age,Occupation\nJohn,25,Engineer\nAlice,30,Doctor\nBob,28,Artist'

# Convert bytes to DataFrame using NumPy and io.BytesIO
array_data = np.genfromtxt(
    BytesIO(bytes_data), delimiter=',', names=True, dtype=None, encoding='utf-8')
df_method2 = pd.DataFrame(array_data)

# Display the DataFrame
print(df_method2)

Output:

    Name  Age Occupation
0   John   25   Engineer
1  Alice   30     Doctor
2    Bob   28     Artist

3. Using Custom Parsing Function

We can use parsing function. here, the code decodes bytes data into strings using UTF-8 encoding, then splits it into records by newline characters. Each record is further split into key-value pairs delimited by ‘|’, and key-value pairs by ‘:’. It constructs dictionaries for each record, with keys and values derived from the splits.

Finally, it assembles these dictionaries into a DataFrame using pandas. This approach allows structured byte data to be converted into a tabular format

import pandas as pd

# Sample bytes data
bytes_data = b'Name:John|Age:25|Occupation:Engineer\nName:Alice|Age:30|Occupation:Doctor\nName:Bob|Age:28|Occupation:Artist'


def parse_bytes_data(data):
    # Decode bytes data and split into records
    records = data.decode('utf-8').split('\n')
    parsed_data = []
    for record in records:
        if record:  # Skip empty records
            items = record.split('|')  # Split record into key-value pairs
            record_dict = {}
            for item in items:
                key, value = item.split(':')  # Split key-value pair
                record_dict[key] = value
            # Append record dictionary to parsed data
            parsed_data.append(record_dict)
    return pd.DataFrame(parsed_data)  # Create DataFrame from parsed data


# Convert bytes to DataFrame using custom parsing function
df_method3 = parse_bytes_data(bytes_data)

# Display the DataFrame
print(df_method3)

Output:

    Name Age Occupation
0   John  25   Engineer
1  Alice  30     Doctor
2    Bob  28     Artist

Conclusion

In conclusion, Python offers different methods for converting bytes to DataFrames like Using the pd.DataFrame Constructor, Using NumPy and io.BytesIO, and Custom Parsing Functions.

Convert Bytes To a Pandas Dataframe – FAQs

How to Convert Data Types in Pandas DataFrame

To convert data types within a pandas DataFrame, you can use the astype() method. This method is versatile and can be applied to individual columns or the entire DataFrame:

import pandas as pd

# Example DataFrame
df = pd.DataFrame({
    'int_column': [1, 2, 3],
    'float_column': [1.1, 2.2, 3.3]
})

# Convert 'float_column' to integer
df['float_column'] = df['float_column'].astype(int)
print(df)

How to Convert Column of Bytes to String in Pandas

If you have a DataFrame column containing data in bytes (b’example’), and you need to convert it to string, you can apply the str.decode() method:

# DataFrame with a bytes column
df = pd.DataFrame({
    'bytes_column': [b'hello', b'world']
})

# Convert bytes to string
df['bytes_column'] = df['bytes_column'].apply(lambda x: x.decode('utf-8'))
print(df)

How to Convert Bytes to String in Python

In Python, converting bytes to a string is done using the decode() method, specifying the encoding (typically ‘utf-8’):

# Example bytes
b_text = b'Hello World'

# Convert bytes to string
text = b_text.decode('utf-8')
print(text)

How to Convert Bytes to Numbers in Python

To convert bytes directly to numbers, you can use Python’s built-in functions depending on the format of the bytes:

# Example bytes representing an integer
b_num = b'\x00\x01'  # Represents the number 1 in 16-bit big-endian

# Convert bytes to integer
num = int.from_bytes(b_num, 'big')
print(num)

How to Convert Integer to String in Pandas DataFrame

To convert an integer to a string within a pandas DataFrame, you again use the astype() method:

# Convert 'int_column' to string
df['int_column'] = df['int_column'].astype(str)
print(df)