{"id":17003,"date":"2020-04-19T06:35:41","date_gmt":"2020-04-19T10:35:41","guid":{"rendered":"https:\/\/bangla.salearningschool.com\/recent-posts\/python-ecommerce-part-2-drop-duplicates-sort-and-take-only-unique-products-after-merging-all-supplier-d-ata-files-into-one-file\/"},"modified":"2020-04-19T15:15:50","modified_gmt":"2020-04-19T19:15:50","slug":"python-ecommerce-part-2-drop-duplicates-sort-and-take-only-unique-products-after-merging-all-supplier-d-ata-files-into-one-file","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=17003","title":{"rendered":"Python: Ecommerce: Part \u2014 2: Drop Duplicates, Sort, and Take Only Unique Products After Merging All Supplier D ata Files into One File"},"content":{"rendered":"
\n
\n
\n
\n
\n

All code in One Block<\/strong><\/p>\n

# # Section: Verify, and Process Supplier Data Before Sending products to \n# # your retail (Magento 2) or marketplace (Amazon, Walmart)<\/span># In[7]:<\/span># combined_csv.sort_values(\u201cModel Code\u201d, inplace = True) \n# dropping ALL duplicte values based on Product SKU = Model Code<\/span>no_duplicates_combined_csv = combined_csv.drop_duplicates(subset = \u201cModel Code\u201d, \n keep = False, inplace = False);\nno_duplicates_combined_csv.shape<\/span># In[8]:<\/span>#55690 vs 55527<\/span># In[9]:<\/span>no_duplicates_combined_csv_verify = combined_csv;\ntype(no_duplicates_combined_csv_verify)<\/span># In[10]:<\/span># verify the shape after dropping duplicates\nno_duplicates_combined_csv_verify.drop_duplicates(subset = \u201cModel Code\u201d, \n keep = False, inplace = True);\nlen(no_duplicates_combined_csv_verify)<\/span># In[11]:<\/span>#55690 vs 55527<\/span># In[12]:<\/span># show combined data : show first 10 rows\nno_duplicates_combined_csv[:3]<\/span># In[16]:<\/span># Stop<\/span># # Find only the unique products, sorted and duplicate removed<\/span># In[14]:<\/span># sorting by SKU = Model Code\nsorted_merged_data = no_duplicates_combined_csv.sort_values(\u201cModel Code\u201d, inplace = False) \nsorted_merged_data.head()<\/span># dropping ALL duplicte values : No need here. Though old code : keeping it anyway\nunique_sorted_data = sorted_merged_data.drop_duplicates(subset =\u201dModel Code\u201d, keep = False, inplace = False) \nunique_sorted_data.head(3)<\/span># In[15]:<\/span># total data count at this point\nunique_sorted_data.shape<\/span><\/pre>\n
From Jupyter Notebook: Cell by Cell with output<\/strong><\/p>\n
Section: Verify, and Process Supplier Data Before Sending products to your retail (Magento 2) or marketplace (Amazon, Walmart)<\/h1>\n
In [7]:<\/p>\n
# combined_csv.sort_values(\"Model Code\", inplace = True)<\/em><\/span># dropping ALL duplicte values based on Product SKU = Model Code<\/em><\/span>no_duplicates_combined_csv =<\/strong> combined_csv.drop_duplicates(subset =<\/strong> \"Model Code\", keep =<\/strong> False<\/strong>, inplace =<\/strong> False<\/strong>);<\/span>no_duplicates_combined_csv.shape<\/span><\/pre>\n
Out[7]:<\/p>\n
(55527, 40)<\/span><\/pre>\n
In [8]:<\/p>\n
#55690 vs 55527<\/em><\/span><\/pre>\n
In [9]:<\/p>\n
no_duplicates_combined_csv_verify =<\/strong> combined_csv;<\/span>type(no_duplicates_combined_csv_verify)<\/span><\/pre>\n
Out[9]:<\/p>\n
pandas.core.frame.DataFrame<\/span><\/pre>\n
In [10]:<\/p>\n
# verify the shape after dropping duplicates<\/em><\/span>no_duplicates_combined_csv_verify.drop_duplicates(subset =<\/strong> \"Model Code\", keep =<\/strong> False<\/strong>, inplace =<\/strong> True<\/strong>);<\/span>len(no_duplicates_combined_csv_verify)<\/span><\/pre>\n
Out[10]:<\/p>\n
55527<\/span><\/pre>\n
In [11]:<\/p>\n
#55690 vs 55527<\/em><\/span><\/pre>\n
In [12]:<\/p>\n
# show combined data : show first 10 rows<\/em><\/span>no_duplicates_combined_csv[:3]<\/span><\/pre>\n
Out[12]:<\/p>\n
Product IDModel CodeFull Product NameShort Product NameProduct URLCategory NameCategory URLSubcategory NameSubcategory URLDate Product Was Launched\u2026Related ProductsRelated AccessoriesWeight KgHeight mmWidth mmDepth mmVideo linkRetail PriceStock statusDate Back0107890POU_0850GV7YPull Rope Fitness Exercises Resistance Bands L\u2026Pull Rope Fitness Exercises Resistance Bands L\u2026<\/p>\n
3 rows \u00d7 40 columns<\/p>\n
In [16]:<\/p>\n
# Stop<\/em><\/span><\/pre>\n
Find only the unique products, sorted and duplicate removed<\/h1>\n
In [14]:<\/p>\n
# sorting by SKU = Model Code<\/em><\/span>sorted_merged_data =<\/strong> no_duplicates_combined_csv.sort_values(\"Model Code\", inplace =<\/strong> False<\/strong>)<\/span>sorted_merged_data.head()<\/span># dropping ALL duplicate values :  No need here. Though old code : keeping it anyway<\/em><\/span>unique_sorted_data =<\/strong> sorted_merged_data.drop_duplicates(subset =<\/strong>\"Model Code\",  keep =<\/strong> False<\/strong>, inplace =<\/strong> False<\/strong>)<\/span>unique_sorted_data.head(3)<\/span><\/pre>\n
Out[14]:<\/p>\n
Product IDModel CodeFull Product NameShort Product NameProduct URLCategory NameCategory URLSubcategory NameSubcategory URLDate Product Was Launched\u2026Related ProductsRelated AccessoriesWeight KgHeight mmWidth mmDepth mmVideo linkRetail PriceStock statusDate Back899230399A01AL3301111Black 3x3x3 MoYu AoLong V2 PuzzleBlack 3x3x3 MoYu AoLong V2 Puzzle<\/p>\n
3 rows \u00d7 40 columns<\/p>\n
In [15]:<\/p>\n
# total data count at this point<\/em><\/span>unique_sorted_data.shape<\/span><\/pre>\n
Out[15]:<\/p>\n
(55527, 40)<\/span><\/pre>\n
***. ***. ***<\/em><\/strong><\/p>\n
Note: Older short-notes from this site are posted on Medium:\u00a0<\/em><\/strong>https:\/\/medium.com\/@SayedAhmedCanada<\/a><\/p>\n
*** . *** *** . *** . *** . ***<\/p>\n
Sayed Ahmed<\/em><\/strong><\/p>\n
BSc. Eng. in Comp. Sc. & Eng. (BUET)<\/em><\/strong><\/p>\n
MSc. in Comp. Sc. (U of Manitoba, Canada)<\/em><\/strong><\/p>\n
MSc. in Data Science and Analytics (Ryerson University, Canada)<\/em><\/strong><\/p>\n
Linkedin<\/em><\/strong>:\u00a0<\/em>https:\/\/ca.linkedin.com\/in\/sayedjustetc<\/em><\/a><\/p>\n
Blog<\/em><\/strong>:\u00a0<\/em>http:\/\/Bangla.SaLearningSchool.com<\/em><\/a>,\u00a0<\/em>http:\/\/SitesTree.com<\/em><\/a><\/p>\n
Training Courses:\u00a0<\/em><\/strong>http:\/\/Training.SitesTree.com<\/em><\/a><\/p>\n
8112223 Canada Inc\/Justetc<\/em><\/strong>:\u00a0<\/em>http:\/\/JustEtc.net<\/em><\/a><\/p>\n
Facebook Groups\/Forums to discuss (Q & A):<\/em><\/strong><\/p>\n
https:\/\/www.facebook.com\/banglasalearningschool<\/a><\/p>\n
https:\/\/www.facebook.com\/justetcsocial<\/a><\/p>\n
Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others.\u00a0<\/em>http:\/\/sitestree.com\/training\/<\/a><\/p>\n<\/div>\n<\/div>\n<\/section>\n<\/div>\n<\/article>\n
\n
\n
\n
\n
\n
\n
Build Ecommerce Software and Systems<\/h2>\n
\n
Build Ecommerce Software and Systems<\/h4>\n<\/div>\n