All code in one block<\/strong><\/p>\n The following is from Jupyter Notebook: Cell By Cell Display. Output data are also shown<\/strong><\/p>\n In [1]:<\/p>\n In [6]:<\/p>\n Out[6]:<\/p>\n In [7]:<\/p>\n Out[7]:<\/p>\n In [8]:<\/p>\n Out[8]:<\/p>\n In [10]:<\/p>\n In [13]:<\/p>\n Out[13]:<\/p>\n Product ID Model Code Full Product NameShort Product NameProduct URLCategory NameCategory URLSubcategory NameSubcategory URLDate Product Was Launched\u2026Related ProductsRelated AccessoriesWeight KgHeight mmWidth mmDepth mmVideo linkRetail PriceStock statusDate Back0107890POU_0850GV7YPull Rope Fitness Exercises Resistance Bands L\u2026Pull Rope Fitness<\/p>\n ***. ***. ***<\/strong><\/em> *** . *** *** . *** . *** . *** Blog<\/strong>: http:\/\/Bangla.SaLearningSchool.com<\/a>, http:\/\/SitesTree.com<\/a><\/em> Facebook Groups\/Forums to discuss (Q & A): <\/strong><\/em> Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. <\/em>http:\/\/sitestree.com\/training\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":" Python: Ecommerce: Part\u200a\u2014\u200a1: Merge Multiple Supplier Data Files into One File Section: Merge multiple Supplier Data Files All code in one block #!\/usr\/bin\/env python # coding: utf-8 # # Section: Merge multiple Supplier Data Files # # In[1]: # if there is a need to merge multiple files \u2014 use this block import os; import … <\/p>\n#!\/usr\/bin\/env python\n# coding: utf-8<\/pre>\n
# # Section: Merge multiple Supplier Data Files\n#<\/pre>\n
# In[1]:<\/pre>\n
# if there is a need to merge multiple files \u2014 use this block\nimport os;\nimport glob;\nimport pandas as pd;<\/pre>\n
# supplier data files\/feeds are kept here\ndata_folder = \u2018data-supplier-2019\u201304\u201314\/supplier-raw-data\/\u2019;\nos.chdir(data_folder);<\/pre>\n
# In[6]:<\/pre>\n
# show all data feed file name\n# file extension for supplier data file\nextension = \u2018csv\u2019;\nall_filenames = [i for i in glob.glob(\u2018*.{}\u2019.format(extension))]\nall_filenames<\/pre>\n# In[7]:<\/pre>\n
# total number of rows combined all data files\/feeds\nrow_total_count = 0\nfor f in all_filenames:\ndf_s = pd.read_csv(f)\nprint(df_s.shape, f)\nrow_total_count += df_s.shape[0]\nrow_total_count # print(row_total_count)<\/pre>\n
# In[8]:<\/pre>\n
# combine all files in the list\ncombined_csv = pd.concat([pd.read_csv(f) for f in all_filenames]);\ncombined_csv.shape<\/pre>\n
# In[10]:<\/pre>\n
# export combined data to a csv file\ncombined_csv.to_csv( \u201c..\/all_supplier_products_2019_04_14.csv\u201d, index=False, encoding=\u2019utf-8-sig\u2019)<\/pre>\n
# In[13]:<\/pre>\n
# read csv data file and show data on the screen\ndf = pd.read_csv(\u2018..\/all_supplier_products_2019_04_14.csv\u2019);\ndf.head()<\/pre>\n
# if there is a need to merge multiple files -- use this block<\/em><\/pre>\n
import<\/strong> os;<\/pre>\n
import<\/strong> glob;<\/pre>\n
import<\/strong> pandas as<\/strong> pd;<\/pre>\n
\n<\/pre>\n
# supplier data files\/feeds are kept here<\/em><\/pre>\n
data_folder =<\/strong> 'data-supplier-2019-04-14\/supplier-raw-data\/';<\/pre>\n
os.chdir(data_folder);<\/pre>\n
# show all data feed file name<\/em><\/pre>\n
# file extension for supplier data file<\/em><\/pre>\n
extension =<\/strong> 'csv';<\/pre>\n
all_filenames =<\/strong> [i for<\/strong> i in<\/strong> glob.glob('*.{}'.format(extension))]<\/pre>\nall_filenames<\/pre>\n
['data_feeds_5e95c25a1f7f6.csv',
'data_feeds_5e95c2962d471.csv',
'data_feeds_5e95c2d255409.csv',
'data_feeds_5e95c30e63423.csv',
'data_feeds_5e95c38646478.csv',
'data_feeds_5e95c5dd76370.csv']<\/pre>\n# total number of rows combined all data files\/feeds<\/em><\/pre>\n
row_total_count =<\/strong> 0<\/pre>\n
for<\/strong> f in<\/strong> all_filenames:<\/pre>\n
df_s =<\/strong> pd.read_csv(f)<\/pre>\n
print(df_s.shape, f)<\/pre>\n
row_total_count +=<\/strong> df_s.shape[0]<\/pre>\n
row_total_count # print(row_total_count)<\/em><\/pre>\n
(8058, 40) data_feeds_5e95c25a1f7f6.csv\n(7, 40) data_feeds_5e95c2962d471.csv\n(1, 40) data_feeds_5e95c2d255409.csv<\/pre>\n
... ....\n(1072, 40) data_feeds_5e95c565d6e30.csv\n(4833, 40) data_feeds_5e95c5dd76370.csv<\/pre>\n
55690<\/pre>\n
# combine all files in the list<\/em><\/pre>\n
combined_csv =<\/strong> pd.concat([pd.read_csv(f) for<\/strong> f in<\/strong> all_filenames]);<\/pre>\n
combined_csv.shape<\/pre>\n
(55690, 40)<\/pre>\n
# export combined data to a csv file<\/em><\/pre>\n
combined_csv.to_csv( "..\/all_supplier_products_2019_04_14.csv", index=False<\/strong>, encoding=<\/strong>'utf-8-sig')<\/pre>\n
df =<\/strong> pd.read_csv('..\/all_supplier_products_2019_04_14.csv');<\/pre>\ndf.head()<\/pre>\n
\nNote: Older short-notes from this site are posted on Medium: <\/strong><\/em>https:\/\/medium.com\/@SayedAhmedCanada<\/a><\/p>\n
\nSayed Ahmed<\/strong>
\n<\/em>
\nBSc. Eng. in Comp. Sc. & Eng. (BUET)<\/strong><\/em>
\nMSc. in Comp. Sc. (U of Manitoba, Canada)<\/strong><\/em>
\nMSc. in Data Science and Analytics (Ryerson University, Canada)<\/strong><\/em>
\nLinkedin<\/strong>: https:\/\/ca.linkedin.com\/in\/sayedjustetc<\/a>
\n<\/em><\/p>\n
\nTraining Courses: <\/strong>http:\/\/Training.SitesTree.com<\/a> <\/em>
\n8112223 Canada Inc\/Justetc<\/strong>: http:\/\/JustEtc.net<\/a><\/em><\/p>\n
\nhttps:\/\/www.facebook.com\/banglasalearningschool<\/a>
\nhttps:\/\/www.facebook.com\/justetcsocial<\/a><\/p>\n