{"id":16994,"date":"2020-04-16T00:04:57","date_gmt":"2020-04-16T04:04:57","guid":{"rendered":"http:\/\/bangla.salearningschool.com\/recent-posts\/python-ecommerce-part%e2%80%8a-%e2%80%8a3-remove-unwanted-category-and-products-also-remove-products-based-on-words-in-the-title%e2%80%8a-%e2%80%8aafter-merging-all-supplier-da\/"},"modified":"2020-04-19T06:49:56","modified_gmt":"2020-04-19T10:49:56","slug":"python-ecommerce-part%e2%80%8a-%e2%80%8a3-remove-unwanted-category-and-products-also-remove-products-based-on-words-in-the-title%e2%80%8a-%e2%80%8aafter-merging-all-supplier-da","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=16994","title":{"rendered":"Python: Ecommerce: Part\u200a\u2014\u200a3: Remove Unwanted Category (and Products), Also, remove products based on Words  in the Title\u200a\u2014\u200aafter Merging All Supplier Data Files into One File"},"content":{"rendered":"<h3>Python: Ecommerce: Part\u200a\u2014\u200a3: Remove Unwanted Category (and Products), Also, remove products based on Words in the Title\u200a\u2014\u200aafter Merging All Supplier Data Files into One File<\/h3>\n<h4>Python: Ecommerce: Part\u200a\u2014\u200a3: Remove Unwanted Category (and Products), Also, remove products based on Words in the Title\u200a\u2014\u200aafter Merging All Supplier Data Files into One File.<\/h4>\n<p> You could as well remove products that are not allowed in a country or in a market place as well as products that you are not authorized to sell (some brands)<\/p>\n<p>All Code in One Block. Please check the other parts of this series\/publication<\/p>\n<p>The code could be simplified\/reduced. You could join multiple blocks into one just by keeping the words\/category names in a list; and then filtering against that list. You could as well join conditions using and (&amp;) or or-operations (|) to reduce the number of lines of code.<\/p>\n<pre># # Section Remove products that have slang words<\/pre>\n<pre># In[24]:<\/pre>\n<pre>unique_sorted_data[\u2018Category Name\u2019].unique()<\/pre>\n<pre># In[30]:<\/pre>\n<pre># Remove products from a category that you do not want to sell<\/pre>\n<pre># Apparel<\/pre>\n<pre>unique_sorted_data_filter_category = unique_sorted_data [<br \/> ~( unique_sorted_data[\u2018Category Name\u2019].str.contains(\u201cApparel\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># android TV Box<\/pre>\n<pre>unique_sorted_data_filter_category = unique_sorted_data_filter_category [<br \/> ~( unique_sorted_data_filter_category[\u2018Category Name\u2019].str.contains(\u201cTV Box\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># Laser Products\nunique_sorted_data_filter_category = unique_sorted_data_filter_category [<br \/> ~( unique_sorted_data_filter_category[\u2018Category Name\u2019].str.contains(\u201cLaser\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># Costume\nunique_sorted_data_filter_category = unique_sorted_data_filter_category [<br \/> ~( unique_sorted_data_filter_category[\u2018Category Name\u2019].str.contains(\u201cCostume\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># Showing the count after removal\nunique_sorted_data_filter_category[\u2018Category Name\u2019].unique(), unique_sorted_data_filter_category.shape<\/pre>\n<pre># In[31]:<\/pre>\n<pre>#################<\/pre>\n<pre># section Remove products that have slang\/bad words in the name -- you do not need this block - was for testing only. Another block will do this job<\/pre>\n<pre>unique_sorted_data_filter_1 = unique_sorted_data[ <br \/> ( unique_sorted_data[\u2018Full Product Name\u2019].str.contains(\u201cSlang 1\u201c, case = False, na=False) )\n\n| ( unique_sorted_data[\u2018Full Product Name\u2019].str.contains(\u201cSlang 2\u201c, case = False, na=False) )\n\n| ( unique_sorted_data[\u2018Full Product Name\u2019].str.contains(\u201cSlang 3\u201c, case = False, na=False) )\n\n| ( unique_sorted_data[\u2018Full Product Name\u2019].str.contains(\u201cSlang 4\u201c, case = False, na=False) )<\/pre>\n<pre>] #[ {\u2018Full Product Name\u2019, \u2018Category Name\u2019}];\nunique_sorted_data_filter_1.shape #, unique_sorted_data_filter_1.head(1) #, \u201c\\n \u201c, unique_sorted_data_filter_1.shape<\/pre>\n<pre>########################<\/pre>\n<pre># In[38]:<\/pre>\n<pre># Remove products that have slang words in the product name<\/pre>\n<pre>unique_sorted_filtered_data = unique_sorted_data_filter_category [<br \/> ~( unique_sorted_data_filter_category[\u2018Full Product Name\u2019].str.contains(\u201cBad word 1\u201c, case = False, na=False ) ) ];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cBad word 2\u201c, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cBad word 3\u201c, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cBad word 4\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cBad word 5\u201c, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cBad word 1\u201c, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cTracker\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cLaser\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># brands that you do not want to sell\nprint(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cVKworld\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>\nprint(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cSamsung\u201d, case = False, na=False ) )\n];<\/pre>\n<pre># video streaming TV Box HDMI -- products are sensitive (intellectual rights)\nprint(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cCar Video\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cStreaming\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data = unique_sorted_filtered_data [<br \/> ~( unique_sorted_filtered_data[\u2018Full Product Name\u2019].str.contains(\u201cHDMI\u201d, case = False, na=False ) )\n];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data.shape<\/pre>\n<pre># In[39]:<\/pre>\n<pre>unique_sorted_filtered_data.to_csv(\u201c..\/all_supplier_data_unique_sorted_and_filtered.csv\u201d);<\/pre>\n<h3><strong>From Jupyter Notebook: Cell by Cell with Output <\/strong><\/h3>\n<h3>Section Remove products that have slang\/bad words or similar. Or Brands\/Products that you do not want to sell (not allowed to sell\/resell\/restricted products or similar): I know the grammar is not right. Sorry, not fixing it.<\/h3>\n<h3>\n<\/h3>\n<p> In [24]:<\/p>\n<pre>unique_sorted_data[&#39;Category Name&#39;].unique()<\/pre>\n<pre>\n<\/pre>\n<p> Out[24]:<\/p>\n<pre>array([&#39;Toys &amp; Games&#39;, &#39;Drone &amp; Quadcopter&#39;, &#39;Cool Gadgets&#39;,<br \/>       &#39;Office supplies&#39;, &#39;Novelty Costumes &amp; Accessories&#39;,<br \/>       &quot;Women&#39;s Jewelry&quot;, &#39;External Parts&#39;, &#39;Vehicle Electronics Devices&#39;,<br \/>       &#39;Replacement Parts&#39;, &#39;Internal Parts&#39;, &#39;Lamps and Accessories&#39;,<br \/>       &#39;Video Games&#39;, &#39;Hair Care&#39;, &#39;Skin Care&#39;,<br \/>       &#39;Makeup Tool &amp; Accessories&#39;, &#39;Household Products&#39;,<br \/>       &quot;Women&#39;s Accessories&quot;, &quot;Women&#39;s Apparel&quot;, &#39;Home accessories&#39;,<br \/>       &#39;Oral Respiratory Protection&#39;, &quot;Men&#39;s Accessories&quot;,<br \/>       &quot;Men&#39;s Apparel&quot;, &quot;Girl&#39;s Apparel&quot;, &#39;Cell Phone Accessories&#39;,<br \/>       &#39;Health Care&#39;, &#39;Electronic Accessories&#39;, &#39;Health tools&#39;,<br \/>       &#39;Computer Peripherals&#39;, &#39;Audio &amp; Video Gadgets&#39;, nan,<br \/>       &#39;Headrest Monitors &amp; DVD Players&#39;, &#39;Car DVR&#39;,<br \/>       &#39;Camera Equipment \/ Accessories&#39;, &#39;Personal Care&#39;,<br \/>       &#39;Laser Gadgets &amp; Measuring Tools&#39;, &#39;Accessories&#39;,<br \/>       &#39;Electronic Cigarettes&#39;, &#39;Sports Action Camera&#39;,<br \/>       &#39;Android TV Box \/ Stick&#39;, &#39;Sports &amp; Body Building&#39;,<br \/>       &#39;Smart Watches&#39;, &#39;Security &amp; Surveillance&#39;, &#39;Android Tablets&#39;,<br \/>       &#39;Musical Instruments &amp; Accessorie&#39;, &#39;LED&#39;, &#39;Outdoor Recrections&#39;,<br \/>       &#39;Tools &amp; Home Decor&#39;, &#39;Home, Kitchen &amp; Garden&#39;, &#39;Home Electrical&#39;,<br \/>       &#39;Bedding &amp; Bath&#39;, &#39;Camping &amp; Hiking&#39;, &#39;Drives &amp; Storage&#39;,<br \/>       &#39;Pet Supplies&#39;, &#39;Hunting &amp; Fishing&#39;, &#39;Garden &amp; Lawn&#39;,<br \/>       &#39;Medical treatments&#39;, &#39;Android Smartphones&#39;, &#39;Car Video&#39;,<br \/>       &#39;Cell Phones&#39;, &#39;Cycling&#39;, &#39;Solar Products&#39;, &#39;Doogee Phones&#39;,<br \/>       &#39;Rugged Phones&#39;, &#39;Ulefone Phones&#39;, &#39;Xiaomi Phones&#39;, &#39;Huawei Phone&#39;,<br \/>       &#39;Lenovo Phones&#39;, &#39;Refurbished iPhones&#39;, &#39;Samsung Phones&#39;,<br \/>       &#39;Water Sport&#39;, &#39;Tools &amp; Equipment&#39;, &#39;Repair Accessories&#39;,<br \/>       &#39;Body protection&#39;, &#39;Disinfection and sterilization&#39;, &quot;Men&#39;s Care&quot;,<br \/>       &#39;Cleaning Supplies&#39;, &#39;Baby Girls Apparel&#39;, &quot;Women&#39;s Bags&quot;,<br \/>       &quot;Women&#39;s Shoes&quot;, &quot;Men&#39;s Jewelry&quot;, &#39;Baby Boys Apparel&#39;,<br \/>       &quot;Boy&#39;s Apparel&quot;, &quot;Girl&#39;s Shoes&quot;, &quot;Girl&#39;s Jewelry&quot;, &quot;Boy&#39;s Shoes&quot;,<br \/>       &#39;kN95\/KF94 Mask&#39;, &#39;Flash Drives + Memory Cards&#39;,<br \/>       &#39;6-7 Inch Android Phones&#39;, &#39;Apple Phones&#39;, &#39;Xiaomi Phone&#39;,<br \/>       &#39;Laptops &amp; Tablets&#39;, &#39;Apple iPad&#39;, &#39;Musical Instruments&#39;,<br \/>       &#39;Computer Accessories&#39;, &#39;Ball Games&#39;, &quot;Boy&#39;s Jewelry&quot;],\ndtype=object)<\/pre>\n<pre>\n<\/pre>\n<p> In [30]:<\/p>\n<pre><em># Remove products from a category that you do not want to sell<\/em><\/pre>\n<pre>unique_sorted_data_filter_category <strong>=<\/strong> unique_sorted_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_data[&#39;Category Name&#39;].str.contains(&quot;Apparel&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>\n<\/pre>\n<pre>unique_sorted_data_filter_category <strong>= <\/strong>unique_sorted_data_filter_category [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_data_filter_category[&#39;Category Name&#39;].str.contains(&quot;TV Box&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>\n<\/pre>\n<pre>unique_sorted_data_filter_category <strong>= <\/strong>unique_sorted_data_filter_category [<strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_data_filter_category[&#39;Category Name&#39;].str.contains(&quot;Laser&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>\n<\/pre>\n<pre>unique_sorted_data_filter_category <strong>=<\/strong> unique_sorted_data_filter_category [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_data_filter_category[&#39;Category Name&#39;].str.contains(&quot;Costume&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )<\/pre>\n<pre>];<\/pre>\n<pre>\n<\/pre>\n<pre>unique_sorted_data_filter_category[&#39;Category Name&#39;].unique(), unique_sorted_data_filter_category.shape<\/pre>\n<p> Out[30]:<\/p>\n<pre>(array([&#39;Toys &amp; Games&#39;, &#39;Drone &amp; Quadcopter&#39;, &#39;Cool Gadgets&#39;,<br \/>        &#39;Office supplies&#39;, &quot;Women&#39;s Jewelry&quot;, &#39;External Parts&#39;,<br \/>        &#39;Vehicle Electronics Devices&#39;, &#39;Replacement Parts&#39;,<br \/>        &#39;Internal Parts&#39;, &#39;Lamps and Accessories&#39;, &#39;Video Games&#39;,<br \/>        &#39;Hair Care&#39;, &#39;Skin Care&#39;, &#39;Makeup Tool &amp; Accessories&#39;,<br \/>        &#39;Household Products&#39;, &quot;Women&#39;s Accessories&quot;, &#39;Home accessories&#39;,<br \/>        &#39;Oral Respiratory Protection&#39;, &quot;Men&#39;s Accessories&quot;,<br \/>        &#39;Cell Phone Accessories&#39;, &#39;Health Care&#39;, &#39;Electronic Accessories&#39;,<br \/>        &#39;Health tools&#39;, &#39;Computer Peripherals&#39;, &#39;Audio &amp; Video Gadgets&#39;,<br \/>        nan, &#39;Headrest Monitors &amp; DVD Players&#39;, &#39;Car DVR&#39;,<br \/>        &#39;Camera Equipment \/ Accessories&#39;, &#39;Personal Care&#39;, &#39;Accessories&#39;,<br \/>        &#39;Electronic Cigarettes&#39;, &#39;Sports Action Camera&#39;,<br \/>        &#39;Sports &amp; Body Building&#39;, &#39;Smart Watches&#39;,<br \/>        &#39;Security &amp; Surveillance&#39;, &#39;Android Tablets&#39;,<br \/>        &#39;Musical Instruments &amp; Accessorie&#39;, &#39;LED&#39;, &#39;Outdoor Recrections&#39;,<br \/>        &#39;Tools &amp; Home Decor&#39;, &#39;Home, Kitchen &amp; Garden&#39;, &#39;Home Electrical&#39;,<br \/>        &#39;Bedding &amp; Bath&#39;, &#39;Camping &amp; Hiking&#39;, &#39;Drives &amp; Storage&#39;,<br \/>        &#39;Pet Supplies&#39;, &#39;Hunting &amp; Fishing&#39;, &#39;Garden &amp; Lawn&#39;,<br \/>        &#39;Medical treatments&#39;, &#39;Android Smartphones&#39;, &#39;Car Video&#39;,<br \/>        &#39;Cell Phones&#39;, &#39;Cycling&#39;, &#39;Solar Products&#39;, &#39;Doogee Phones&#39;,<br \/>        &#39;Rugged Phones&#39;, &#39;Ulefone Phones&#39;, &#39;Xiaomi Phones&#39;, &#39;Huawei Phone&#39;,<br \/>        &#39;Lenovo Phones&#39;, &#39;Refurbished iPhones&#39;, &#39;Samsung Phones&#39;,<br \/>        &#39;Water Sport&#39;, &#39;Tools &amp; Equipment&#39;, &#39;Repair Accessories&#39;,<br \/>        &#39;Body protection&#39;, &#39;Disinfection and sterilization&#39;, &quot;Men&#39;s Care&quot;,<br \/>        &#39;Cleaning Supplies&#39;, &quot;Women&#39;s Bags&quot;, &quot;Women&#39;s Shoes&quot;,<br \/>        &quot;Men&#39;s Jewelry&quot;, &quot;Girl&#39;s Shoes&quot;, &quot;Girl&#39;s Jewelry&quot;, &quot;Boy&#39;s Shoes&quot;,<br \/>        &#39;kN95\/KF94 Mask&#39;, &#39;Flash Drives + Memory Cards&#39;,<br \/>        &#39;6-7 Inch Android Phones&#39;, &#39;Apple Phones&#39;, &#39;Xiaomi Phone&#39;,<br \/>        &#39;Laptops &amp; Tablets&#39;, &#39;Apple iPad&#39;, &#39;Musical Instruments&#39;,<br \/>        &#39;Computer Accessories&#39;, &#39;Ball Games&#39;, &quot;Boy&#39;s Jewelry&quot;],\ndtype=object), (47826, 40))<\/pre>\n<pre>\n<\/pre>\n<p> In [31]:<\/p>\n<pre><em># sect<\/em><\/pre>\n<pre>\n<\/pre>\n<p> In [38]:<\/p>\n<pre><em># Remove products that have slang words in the product name<\/em><\/pre>\n<pre>unique_sorted_filtered_data <strong>=<\/strong> unique_sorted_data_filter_category [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_data_filter_category[&#39;Full Product Name&#39;].str.contains(&quot;Bad word 1&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>\n<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre>unique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [<strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Bad word 2&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [<strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Bad word 3&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre># product\nprint(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Tracker&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) ) ];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Laser&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre># remove brands\nunique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;VKworld&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre>unique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Samsung&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre># video, streaming, HDMI\nunique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Car Video&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre>unique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [ <strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;Streaming&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);<\/pre>\n<pre>unique_sorted_filtered_data <strong>=<\/strong> unique_sorted_filtered_data [<strong class=\"gmail-markup--strong gmail-markup--pre-strong\">~<\/strong>( unique_sorted_filtered_data[&#39;Full Product Name&#39;].str.contains(&quot;HDMI&quot;, case <strong>=<\/strong> <strong>False<\/strong>, na<strong>=False<\/strong> ) )];<\/pre>\n<pre>print(unique_sorted_filtered_data.shape);\nunique_sorted_filtered_data.shape<\/pre>\n<pre>(47001, 40)\n(46988, 40)\n(46968, 40)\n(46968, 40)\n(46959, 40)\n(46959, 40)\n(46441, 40)\n(46388, 40)\n(46381, 40)\n(45357, 40)\n(45349, 40)\n(45338, 40)\n(44911, 40)<\/pre>\n<p> Out[38]:<\/p>\n<pre>(44911, 40)<\/pre>\n<p> In [39]:<\/p>\n<pre># send the filtered data to a file\nunique_sorted_filtered_data.to_csv(&quot;..\/all_supplier_data_unique_sorted_and_filtered.csv&quot;);<\/pre>\n<p><em><strong>***. ***. ***<\/strong><\/em><br \/>\n<em><strong>Note: Older short-notes from this site are posted on Medium: <\/strong><\/em><a href=\"https:\/\/medium.com\/@SayedAhmedCanada\">https:\/\/medium.com\/@SayedAhmedCanada<\/a><\/p>\n<p>*** . *** *** . *** . *** . ***<br \/>\n<em><strong>Sayed Ahmed<\/strong><br \/>\n<\/em><br \/>\n<em><strong>BSc. Eng. in Comp. Sc. &amp; Eng. (BUET)<\/strong><\/em><br \/>\n<em><strong>MSc. in Comp. Sc. (U of Manitoba, Canada)<\/strong><\/em><br \/>\n<em><strong>MSc. in Data Science and Analytics (Ryerson University, Canada)<\/strong><\/em><br \/>\n<em><strong>Linkedin<\/strong>: <a href=\"https:\/\/ca.linkedin.com\/in\/sayedjustetc\">https:\/\/ca.linkedin.com\/in\/sayedjustetc<\/a><br \/>\n<\/em><\/p>\n<p><em><strong>Blog<\/strong>: <a href=\"http:\/\/bangla.salearningschool.com\/\">http:\/\/Bangla.SaLearningSchool.com<\/a>, <a href=\"http:\/\/sitestree.com\">http:\/\/SitesTree.com<\/a><\/em><br \/>\n<em><strong>Training Courses: <\/strong><a href=\"http:\/\/training.SitesTree.com\">http:\/\/Training.SitesTree.com<\/a> <\/em><br \/>\n<em><strong>8112223 Canada Inc\/Justetc<\/strong>: <a href=\"http:\/\/JustEtc.net\">http:\/\/JustEtc.net<\/a><\/em><\/p>\n<p><em><strong>Facebook Groups\/Forums to discuss (Q &amp; A): <\/strong><\/em><br \/>\n<a href=\"https:\/\/www.facebook.com\/banglasalearningschool\">https:\/\/www.facebook.com\/banglasalearningschool<\/a><br \/>\n<a href=\"https:\/\/www.facebook.com\/justetcsocial\">https:\/\/www.facebook.com\/justetcsocial<\/a><\/p>\n<p><em>Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. <\/em><a href=\"http:\/\/sitestree.com\/training\/\">http:\/\/sitestree.com\/training\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Python: Ecommerce: Part\u200a\u2014\u200a3: Remove Unwanted Category (and Products), Also, remove products based on Words in the Title\u200a\u2014\u200aafter Merging All Supplier Data Files into One File Python: Ecommerce: Part\u200a\u2014\u200a3: Remove Unwanted Category (and Products), Also, remove products based on Words in the Title\u200a\u2014\u200aafter Merging All Supplier Data Files into One File. You could as well remove &hellip; <\/p>\n<p><a class=\"more-link btn\" href=\"http:\/\/bangla.sitestree.com\/?p=16994\">Continue reading<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[1912,182],"tags":[],"class_list":["post-16994","post","type-post","status-publish","format-standard","hentry","category-build-ecommerce-software","category---blog","item-wrap"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":17004,"url":"http:\/\/bangla.sitestree.com\/?p=17004","url_meta":{"origin":16994,"position":0},"title":"Python: Ecommerce: Part \u2014 3: Remove Unwanted Category (and Products), Also, remove products based on Words in  the Title \u2014 after Merging All Supplier Data Files into One File","author":"Sayed","date":"April 19, 2020","format":false,"excerpt":"You could as well remove products that are not allowed in a country or in a market place as well as products that you are not authorized to sell (some brands) All Code in One Block. Please check the other parts of this series\/publication The code could be simplified\/reduced. You\u2026","rel":"","context":"In &quot;Build Ecommerce Software&quot;","block_context":{"text":"Build Ecommerce Software","link":"http:\/\/bangla.sitestree.com\/?cat=1912"},"img":{"alt_text":"8112223 Canada Inc. (Justetc)","src":"https:\/\/miro.medium.com\/fit\/c\/80\/80\/0*P_esmjKoJnHlNjFX","width":350,"height":200},"classes":[]},{"id":17003,"url":"http:\/\/bangla.sitestree.com\/?p=17003","url_meta":{"origin":16994,"position":1},"title":"Python: Ecommerce: Part \u2014 2: Drop Duplicates, Sort, and Take Only Unique Products After Merging All Supplier D ata Files into One File","author":"Sayed","date":"April 19, 2020","format":false,"excerpt":"All code in One Block # # Section: Verify, and Process Supplier Data Before Sending products to # # your retail (Magento 2) or marketplace (Amazon, Walmart)# In[7]:# combined_csv.sort_values(\u201cModel Code\u201d, inplace = True) # dropping ALL duplicte values based on Product SKU = Model Codeno_duplicates_combined_csv = combined_csv.drop_duplicates(subset = \u201cModel Code\u201d,\u2026","rel":"","context":"In &quot;Build Ecommerce Software&quot;","block_context":{"text":"Build Ecommerce Software","link":"http:\/\/bangla.sitestree.com\/?cat=1912"},"img":{"alt_text":"8112223 Canada Inc. (Justetc)","src":"https:\/\/miro.medium.com\/fit\/c\/80\/80\/0*P_esmjKoJnHlNjFX","width":350,"height":200},"classes":[]},{"id":17002,"url":"http:\/\/bangla.sitestree.com\/?p=17002","url_meta":{"origin":16994,"position":2},"title":"Python: Ecommerce: Part \u2014 1: Merge Multiple Supplier Data Files into One File","author":"Sayed","date":"April 19, 2020","format":false,"excerpt":"Section: Merge multiple Supplier Data Files All code in one block #!\/usr\/bin\/env python # coding: utf-8# # Section: Merge multiple Supplier Data Files ## In[1]:# if there is a need to merge multiple files \u2014 use this block import os; import glob; import pandas as pd;# supplier data files\/feeds are\u2026","rel":"","context":"In &quot;Build Ecommerce Software&quot;","block_context":{"text":"Build Ecommerce Software","link":"http:\/\/bangla.sitestree.com\/?cat=1912"},"img":{"alt_text":"8112223 Canada Inc. (Justetc)","src":"https:\/\/miro.medium.com\/fit\/c\/80\/80\/0*P_esmjKoJnHlNjFX","width":350,"height":200},"classes":[]},{"id":17001,"url":"http:\/\/bangla.sitestree.com\/?p=17001","url_meta":{"origin":16994,"position":3},"title":"Python: Ecommerce: Part\u200a\u2014\u200a1: Merge Multiple Supplier Data Files into One File","author":"Sayed","date":"April 18, 2020","format":false,"excerpt":"Python: Ecommerce: Part\u200a\u2014\u200a1: Merge Multiple Supplier Data Files into One File Section: Merge multiple Supplier Data Files All code in one block #!\/usr\/bin\/env python # coding: utf-8 # # Section: Merge multiple Supplier Data Files # # In[1]: # if there is a need to merge multiple files \u2014 use\u2026","rel":"","context":"In &quot;Build Ecommerce Software&quot;","block_context":{"text":"Build Ecommerce Software","link":"http:\/\/bangla.sitestree.com\/?cat=1912"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":17008,"url":"http:\/\/bangla.sitestree.com\/?p=17008","url_meta":{"origin":16994,"position":4},"title":"Python: Ecommerce: Part \u2014 7: Partition a Data File (with product information) into Multiple Files.","author":"Sayed","date":"April 19, 2020","format":false,"excerpt":"In previous steps, we created a product data file to be uploaded to Magento 2 (from supplier data). However, there is a limit, how big a file can be uploaded to Magento 2 for product data import. Hence, this code will divide the data file into multiple files. This data\u2026","rel":"","context":"In &quot;Build Ecommerce Software&quot;","block_context":{"text":"Build Ecommerce Software","link":"http:\/\/bangla.sitestree.com\/?cat=1912"},"img":{"alt_text":"8112223 Canada Inc. (Justetc)","src":"https:\/\/miro.medium.com\/fit\/c\/80\/80\/0*P_esmjKoJnHlNjFX","width":350,"height":200},"classes":[]},{"id":16964,"url":"http:\/\/bangla.sitestree.com\/?p=16964","url_meta":{"origin":16994,"position":5},"title":"Python: Merge Multiple csv files into one to facilitate reporting on transaction data over time","author":"Sayed","date":"March 21, 2020","format":false,"excerpt":"Python: Merge Multiple csv files into one to facilitate reporting on transaction data over time By Sayed Ahmed Merge multiple transaction files into one. This is an extension to the article: Python: Read RBC Canada: Mastercard PDF Statement Transaction Data into CSV file By Sayed Ahmedmedium.com The Code for Merging\u2026","rel":"","context":"In &quot;\u09ac\u09cd\u09b2\u0997 \u0964 Blog&quot;","block_context":{"text":"\u09ac\u09cd\u09b2\u0997 \u0964 Blog","link":"http:\/\/bangla.sitestree.com\/?cat=182"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16994","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=16994"}],"version-history":[{"count":1,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16994\/revisions"}],"predecessor-version":[{"id":17021,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/16994\/revisions\/17021"}],"wp:attachment":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=16994"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=16994"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=16994"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}