Can you answer these questions on Data Science Project Development

Questions to answer

1. What does a data science project usually involve? What is the common theme across data science projects?

2. Does industry projects and research projects differ? Why and to what extent?

3. What are the some dataset repositories? Where can you get them?

4. Are all public datasets perfect for doing a project?

5. Can you get one dataset with all required data?

6. if you have different datasets with part of the data that you need — what do you do? do you just abandon the project?

7. True or false. you get a dataset and then directly apply your algorithm to get the answer or to predict?

8. Is there any benefit to check the data details such as see the distribution of the data? mean and median and mode of the data?

9. What are the usual steps in a data science research project?

10. what is univariate analysis?

11. what is bivariate analysis?

12. is multivariate analysis?

13. Why do you do univariate, bivariate or multivariate analysis?

14. What is exploratory data analysis?

15. What are the two types of exploratory data analysis.

16. What are the pros and cons of visualizing the raw data?

17. Is there any need to use clustering while doing data exploration? or this can be more of a methodology of the research process?

18. what is data synthesis?

19. What is study selection?

20. What do the materials and methods section in a data science research project contain?

21. Difference between Methodology and experiment?

22. Difference between Exploratory analysis step and methodology step?

23. What makes a good experiment?

24. what is a regression analysis? Can it be an exploratory analysis? can it be as part of experimental analysis.

25. what is PCA?

26. What is Principal Component Analysis?

27. What is factor analysis?

28. True or false => PCA and Factor analysis do the same thing i,e. they are the same? why, why not explain

29. What is the measure name that indicates whether factor analysis is required or not

30. With PCA, you can always ignore the similar behaving/contributing features?

31. can you find out the purpose of decision trees, SVM, deep learning. is there an opportunity to apply those for the project in concern? to what extent and how? is ther any more work to do for the dataset preparation?

32. Can you write Python code to visualize a feature with a box plot? If not, can you google/startpage to find out the code and implement? What does the Boxplot for this feature saying?

33. what are the prediction methodology used in the project? Which one performed the best? Does the result make sense? Does the perfect result make sense? Do you think there can be bias and the code will not be able to give similar output for other datasets?

34. What are the measures used for prediction performance?

35. What was used for the project: Linear/Polynomial Regression or Logistics Regression. What is a better measure between these two for this project or in general?

36. True or false, Logistic regression is for Regression

37. True or false, Linear regression is for Classification

38. What is the significance of R square? Low or high values of R square is desirable to say that the Regression outcome is important?

39. What is f-score?

40. What is the train-test-split method in Python?

41. Did I use SQL in the project? Was that a must? What else could you do?

42. Can you join multiple dataframe (i.e. table) in Python? Do you have left, right, inner or similar joins in Python.

(When I originally wrote) Last modified: Saturday, 9 November 2019, 4:10 PM

In Object Oriented Design:

What are Association, Aggregation, Composition, and Generalization?

How are these represented in Diagrams? Draw the diagrams.

Give examples of Association, Aggregation, Composition, and Generalization. Write some example skeleton classes in any OOP language.

What is the difference between Aggregation and composition?

What is the difference between  Generalization and Specialization?

Ref: https://www.visual-paradigm.com/guide/uml-unified-modeling-language/uml-aggregation-vs-composition/

Aggregation and Composition are subsets of association meaning they are specific cases of association. In both aggregation and composition object of one class “owns” object of another class. But there is a subtle difference:

• Aggregation implies a relationship where the child can exist independently of the parent. Example: Class (parent) and Student (child). Delete the Class and the Students still exist.
• Composition implies a relationship where the child cannot exist independent of the parent. Example: House (parent) and Room (child). Rooms don’t exist separate to a House.

“Generalization is a mechanism for combining similar classes of objects into a single, more general class.”

“Specialization is the reverse process of Generalization means creating new sub-classes from an existing class.”

Adapter Design Patterns

What is a design pattern?

What is a Structural Design Pattern?

What is an Adapter Design Pattern?

What is the purpose of the Adapter Design Pattern?

What is the category of the Adapter Design Pattern?

Provide use cases where you can use a Adapter design pattern.

How do you implement the Adapter design pattern?

Write example classes to explain Adapter design pattern?

Adapter is a structural design pattern that allows objects with incompatible interfaces to collaborate.

Ref: A good read: https://refactoring.guru/design-patterns/adapter

Ref: Wikipedia

What is a design pattern?

What is a Structural Design Pattern?

What is an Adapter Design Pattern?

What is the purpose of the Adapter Design Pattern?

What is the category of the Adapter Design Pattern?

Provide use cases where you can use a Adapter design pattern.

How do you implement the Adapter design pattern?

Write example classes to explain Adapter design pattern?

db.Shirts.remove({});

db.Shirts.insertMany(
    [
        {
            Brand:"Calvin",
            Name: "Name 1",
            Color: ["Blue", "Red"],
            Price: 100,
            Size: {
                height:10, width:5
            }
        },
        
        {
            Brand:"Tommy",
            Name: "Name 2",
            Color: ["Blue", "Red", "Orange"],
            Price: 10,
            Size: {
                height:10, width:5
            }
        }
        
        ,
        
        {
            Brand:"Calvin Kleins",
            Name: "Name 3",
            Color: ["Blue", "Red", "Orange"],
            Price: 50,
            Size: {
                height:10, width:5
            }
        }
        
        
    ]
  );
  
  // related to: Update the Brand to Calvin Klein when you see only Calvin  
  db.Shirts.find( {Brand:"Calvin"} );
  db.Shirts.find({Brand:{regex:/Calvin/}});
  db.Shirts.find({Brand:"Calvin"});  
  db.Shirts.find({Brand:"Tommy"});  
  db.Shirts.updateMany({Brand:"Calvin"}, {$set:{Brand:"Calvin Klein"}} );
  db.Shirts.find({});
  db.Shirts.find( 
      {
         Brand:"Calvin"
      } 
  );
  
  db.Shirts.find( {
      Brand:{
              $gt:2
          }
      } 
  );
  
  // https://www.mongodb.com/docs/manual/reference/operator/query/regex/
  db.Shirts.find({            
       Brand: { $regex: /Cal/ }       
  });
  
  // Find Shirts that costs between 50 and 100  
  db.Shirts.find(
      {
          Price:
          {
              $gte:50, $lte:100
          }
      }
  );
  
  // Find Shirts that costs between 50 and 100  
  db.Shirts.find(
      {
          $and:[
              {Price:{$gte:50}},                  
              {Price:{$lte:100}}          
          ]                   
      }
  );
  
  
  // Find shirts where the color is Blue or Red
  db.Shirts.find (
      {
          Color: {$in:["Red", "Blue"]}
      }
  );
  
  
db.Shirts.find({"_id" : ObjectId("660191c185c168d7ae91df2f")});
db.collection.findById('4ecbe7f9e8c1c9092c000027');

let id = '660191c185c168d7ae91df2f';
let o_id = new ObjectId(id);
db.Shirts.findOne({"_id":o_id});



db.runCommand( 
    {
       create: "Shirts",
       clusteredIndex: { 
           "key": { _id: 1 }, 
           "unique": true, 
           "name": "ShirtsClusteredKey" 
      }
    } 

);



db.movies.aggregate([
  {
    $search: {
      index: "default", // optional unless you named your index something other than "default"
      text: {
        query: "star wars",
        path: "title"
      },
    },
  },
  {
    $project: {
      title: 1,
      year: 1,
    }
  }
])


db.runCommand( {
   create: "Shirts",
   clusteredIndex: { "key": { price: 1 }, "unique": true, "name": "products clustered key" }
} )


db.Shirts.createIndex( { Price:1 } )
db.Shirts.aggregate([
  {
    $search: {
      index: "Price", // optional unless you named your index something other than "default"
      text: {
        query: "Tommy",
        path: "Brand"
      },
    },
  },
  {
    $project: {
      Brand: 1,
      Price: 1,
    }
  }
])


db.Shirts.getIndexes()
db.Shirts.find({Price:{$gt:10}});

var shrt = db.Shirts.find({});
while (shrt.hasNext() ) {
    print(tojson(shrt.next()))
}

var shrt = db.Shirts.find({});
print(shrt);
while (shrt.hasNext() ) {
    print(shrt.next())
}



db.Reviews.aggregate([
  {
    $lookup: {
      from: "Products",
      localField: "product_id",
      foreignField: "_id",
      as: "movie_details",
    },
  },
  {
    $limit: 1
  }
])





  
  

db.Shirts.remove({});

db.Shirts.drop({});

db.Shirts.insertMany(
    [
        {
            Brand:"Calvin",
            Name: "Name 1",
            Color: ["Blue", "Red"],
            Price: 100,
            Size: {
                height:10, width:5
            }
        },
        
        {
            Brand:"Tommy",
            Name: "Name 2",
            Color: ["Blue", "Red", "Orange"],
            Price: 10,
            Size: {
                height:10, width:5
            }
        }
        
        ,
        
        {
            Brand:"Calvin Kleins",
            Name: "Name 3",
            Color: ["Blue", "Red", "Orange"],
            Price: 50,
            Size: {
                height:10, width:5
            }
        }
        
        
    ]
  );
  
  // related to: Update the Brand to Calvin Klein when you see only Calvin  
  db.Shirts.find( {Brand:"Calvin"} );
  db.Shirts.find({Brand:{regex:/Calvin/}});
  db.Shirts.find({Brand:"Calvin"});  
  db.Shirts.find({Brand:"Tommy"});  
  db.Shirts.updateMany({Brand:"Calvin"}, {$set:{Brand:"Calvin Klein"}} );
  db.Shirts.find({});
  db.Shirts.find( 
      {
         Brand:"Calvin"
      } 
  );
  
  db.Shirts.find( {
      Brand:{
              $gt:2
          }
      } 
  );
  
  // https://www.mongodb.com/docs/manual/reference/operator/query/regex/
  db.Shirts.find({            
       Brand: { $regex: /Cal/ }       
  });
  
  // Find Shirts that costs between 50 and 100  
  db.Shirts.find(
      {
          Price:
          {
              $gte:50, $lte:100
          }
      }
  );
  
  // Find Shirts that costs between 50 and 100  
  db.Shirts.find(
      {
          $and:[
              {Price:{$gte:50}},                  
              {Price:{$lte:100}}          
          ]                   
      }
  );
  
  
  // Find shirts where the color is Blue or Red
  db.Shirts.find (
      {
          Color: {$in:["Red", "Blue"]}
      }
  );
  

db.orders.insertMany( [
   { “_id” : 1, “item” : “almonds”, “price” : 12, “quantity” : 2 },
   { “_id” : 2, “item” : “pecans”, “price” : 20, “quantity” : 1 },
   { “_id” : 3  }
] )

db.inventory.insertMany( [
   { “_id” : 1, “sku” : “almonds”, “description”: “product 1”, “instock” : 120 },
   { “_id” : 2, “sku” : “bread”, “description”: “product 2”, “instock” : 80 },
   { “_id” : 3, “sku” : “cashews”, “description”: “product 3”, “instock” : 60 },
   { “_id” : 4, “sku” : “pecans”, “description”: “product 4”, “instock” : 70 },
   { “_id” : 5, “sku”: null, “description”: “Incomplete” },
   { “_id” : 6 }
] );

db.sales.insertMany([
  { “_id” : 1, “item” : “abc”, “price” : Decimal128(“10”), “quantity” : Int32(“2”), “date” : ISODate(“2014-03-01T08:00:00Z”) },
  { “_id” : 2, “item” : “jkl”, “price” : Decimal128(“20”), “quantity” : Int32(“1”), “date” : ISODate(“2014-03-01T09:00:00Z”) },
  { “_id” : 3, “item” : “xyz”, “price” : Decimal128(“5”), “quantity” : Int32( “10”), “date” : ISODate(“2014-03-15T09:00:00Z”) },
  { “_id” : 4, “item” : “xyz”, “price” : Decimal128(“5”), “quantity” :  Int32(“20”) , “date” : ISODate(“2014-04-04T11:21:39.736Z”) },
  { “_id” : 5, “item” : “abc”, “price” : Decimal128(“10”), “quantity” : Int32(“10”) , “date” : ISODate(“2014-04-04T21:23:13.331Z”) },
  { “_id” : 6, “item” : “def”, “price” : Decimal128(“7.5”), “quantity”: Int32(“5” ) , “date” : ISODate(“2015-06-04T05:08:13Z”) },
  { “_id” : 7, “item” : “def”, “price” : Decimal128(“7.5”), “quantity”: Int32(“10”) , “date” : ISODate(“2015-09-10T08:43:00Z”) },
  { “_id” : 8, “item” : “abc”, “price” : Decimal128(“10”), “quantity” : Int32(“5” ) , “date” : ISODate(“2016-02-06T20:20:13Z”) },
])

db.orders.aggregate( [
   {
     $lookup:
       {
         from: “inventory”,
         localField: “item”,
         foreignField: “sku”,
         as: “inventory_docs”
       }
  }
] )

db.Products.aggregate( [
   {
      $lookup:
         {
            from: “Reviews”,
            localField: “ReviewIds”,
            foreignField: “_id”,
            as: “enrollee_info”
        }
   }
] )

db.Products.aggregate(
[
    {
        $lookup: {
            from: “Products”,
            localField: “ReviewIds”,
            foreignField: “_id”,
            as: “movie_details”
        }
    }]
);

db.Reviews.drop();

db.Reviews.insertMany([
 
    {_id:1, rating:5, comment: "overall not bad"},
    {_id:2, rating:2, comment: "overall not bad"},
    {_id:3, rating:3, comment: "overall not bad"},
    {_id:4, rating:5, comment: "overall not bad"},
    {_id:5, rating:4, comment: "overall not bad"},
    {_id:6, rating:5, comment: "overall not bad"}
 
]);  

db.ProductRefModel.drop(); 
db.ProductRefModel.insertMany([
    {
        ProductId:1,
        Description: "Bottle",
        ReviewIds: [ 1, 2 ]
    },
    {
        ProductId: 2,
        Description: "Air Pump",
        ReviewIds: [3,4]
    }
    ]);
    
    
db.ProductRefModel.find();
 
db.ProductRefModel.aggregate([
    {
        $lookup: {
            from: "Reviews",
            localField: "ReviewIds",
            foreignField: "_id",
            as: "Rating Details"
        }
    }
  ]
); 


db.Reviews.aggregate([
    {
        $lookup: {
            from: "ProductRefModel",
            localField: "_id",
            foreignField: "ReviewIds",
            as: "Product Details"
        }
    }
  ]
);