Skip to main content

Posts

Showing posts with the label Pandas

Read data from a dictionary in pandas and assign new column value

I have dict data in a pandas column and I am attempting to assign values to a new column from the dict . import pandas as pd df = pd.DataFrame({'n': [np.nan, np.nan], 'd': [{'status': {'version': '1.0.0'}, 'p': [{'identifier': {'Id': 5155, 'fips': '07', 'aId': 5175}, 'lt': {'ltnum': '14', 'ls1': 0.1485537, 'ls2': 6471, 'ptype': 'NO'}}]}, np.nan] }) When I try: df['n'].fillna(df['d'].str['p'][0][0]['lt']['ls1'], inplace=True) I get a: TypeError: 'float' object is not subscriptable by kms in StackOverflow on July 03, 20...

How to parse pandas column in the loop if it's a dictionary

I have the DataFrame, one of the column contains dictionary in rows in format: rates 0 {'time': '2022-06-05T19:25:57.3000000Z', 'asset_id_quote': '0X', 'rate': 73571.98764837519} 1 {'time': '2022-06-05T19:25:57.3000000Z', 'asset_id_quote': '18C', 'rate': 11607635.869234081} 2 {'time': '2022-06-05T19:25:57.3000000Z', 'asset_id_quote': '1EARTH', 'rate': 4162937.862828232} 3 {'time': '2022-06-05T19:25:57.3000000Z', 'asset_id_quote': '1ECO', 'rate': 7337.697146675354} 4 {'time': '2022-06-05T19:25:57.3000000Z', 'asset_id_quote': '1INCH', 'rate': 34645.31024605586} ... ... 49...

Finding a mismatch between two correlated columns in pandas dataframe

I have a large pandas dataframe which has two columns. One column is the unique code for an item and the second column contains the name of that item. For example purposes lets say 'ID' has a unique code for an animal and 'Name' has the name of that animal. df = pd.DataFrame({'ID': ['AA','BB', 'CC', 'DD', 'EE'], 'Name': ['Ape', 'Bull', 'Chimp', 'Dog', 'Dog']}) ID Name 0 AA Ape 1 BB Bull 2 CC Chimp 3 DD Dog 4 EE Dog When I run the code below I will get false because there is one more unique item in the ID column than in the Name column len(df['Code'].unique()) == len(df['Name'].unique()) My question is how can I find out which animal is being represented by two codes. Ideally I would like to end up with a dataframe that has: ID Name 3 DD Dog 4 EE Dog But as long as I can ide...

Looping over vectors in python (analogous to Pandas columns)

I have a number of analogous operations that I would like to perform on different vectors. What is the 'pythonesque' way of 'looping' over these variables to save lines? import numpy as np length = 10 l = np.ones(length, dtype = int) r = np.ones(length, dtype = int) v = np.ones(length, dtype = int) w = np.ones(length, dtype = int) q = np.ones(length, dtype = int) new_l = np.ones(length, dtype = int) new_r = np.ones(length, dtype = int) new_v = np.ones(length, dtype = int) new_w = np.ones(length, dtype = int) new_q = np.ones(length, dtype = int) l = 0.05 * new_l + 0.95 * l r = 0.05 * new_r + 0.95 * r v = 0.05 * new_v + 0.95 * v w = 0.05 * new_w + 0.95 * w q = 0.05 * new_q + 0.95 * q If these were columns in a Pandas dataframe, I would write: import pandas as pd length = 10 df = pd.DataFrame(index = range(length)) for v in ['l', 'r', 'v', 'w', 'q']: df[f'{v}'] = 1 df[f'new_{v}'] = 1 df[...