{"id":76091,"date":"2024-05-19T23:01:21","date_gmt":"2024-05-20T03:01:21","guid":{"rendered":"https:\/\/bangla.sitestree.com\/spearman-correlation-coefficient-and-graph-mining\/"},"modified":"2024-05-19T23:01:21","modified_gmt":"2024-05-20T03:01:21","slug":"spearman-correlation-coefficient-and-graph-mining","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=76091","title":{"rendered":"Spearman Correlation Coefficient and Graph Mining"},"content":{"rendered":"<p>#!\/usr\/bin\/env python<\/p>\n<h1>coding: utf-8<\/h1>\n<h1># 3rd Model: Deepgraph CNN: Stock Price Prediction using DeepGraphCNN Neural Networks. It includes GCN layers and CNN layers. I have added an MLP at the last layer to predict stock prices.<\/h1>\n<p>#<\/p>\n<h1># Input graphs were created for spearman, Spearman, and Kendal Tau correlations\/coefficients from historical stock prices. Also, another graph is created based on financial news articles.<\/h1>\n<p>#<\/p>\n<h1># For the sake of making execution easier (and at once), I have kept multiple approaches (spearman, Spearman, and Kendal Tau, News Based) in the same file. One big code file can be difficult to handle; is done just for making execution easier.<\/h1>\n<p>#<\/p>\n<h1># Because I initially tried separately and brought the code together, some code might be a bit redundant\/repeating. I may have done some cleaning.<\/h1>\n<p>#<\/p>\n<h1># An use case of DeepGraphCNN for Node Classification<\/h1>\n<h1># <a href=\"https:\/\/stellargraph.readthedocs.io\/en\/latest\/demos\/graph-classification\/dgcnn-graph-classification.html\">https:\/\/stellargraph.readthedocs.io\/en\/latest\/demos\/graph-classification\/dgcnn-graph-classification.html<\/a><\/h1>\n<p>#<\/p>\n<h1># Import Libraries<\/h1>\n<h1>In[1]:<\/h1>\n<h1>import libraries<\/h1>\n<p>import os<br \/>\nimport pandas as pd<br \/>\nimport math<\/p>\n<h1>In[2]:<\/h1>\n<h1>Import Libraries for Graph, GNN, and GCN<\/h1>\n<p>import stellargraph as sg<br \/>\nfrom stellargraph import StellarGraph<br \/>\nfrom stellargraph.layer import DeepGraphCNN<br \/>\nfrom stellargraph.mapper import FullBatchNodeGenerator<br \/>\nfrom stellargraph.mapper import PaddedGraphGenerator<br \/>\nfrom stellargraph.layer import GCN<\/p>\n<h1>In[3]:<\/h1>\n<h1>Machine Learnig related library Imports<\/h1>\n<p>from tensorflow.keras import layers, optimizers, losses, metrics, Model<br \/>\nfrom sklearn import preprocessing, model_selection<br \/>\nfrom IPython.display import display, HTML<br \/>\nimport matplotlib.pyplot as plt<br \/>\nget_ipython().run_line_magic(&#8216;matplotlib&#8217;, &#8216;inline&#8217;)<br \/>\nfrom tensorflow.keras.layers import Dense, Conv1D, MaxPool1D, Dropout, Flatten<br \/>\nfrom tensorflow import keras<\/p>\n<h1>In[4]:<\/h1>\n<h1>If we want to drop NAN column or row wise for stock price data<\/h1>\n<h1>I did not need to use this options that much<\/h1>\n<p>drop_cols_with_na = 1<br \/>\ndrop_rows_with_na = 1<\/p>\n<h1># Dataset: Using 30 companies from the Fortune 500 companies (the paper used these stocks)<\/h1>\n<h1>In[5]:<\/h1>\n<p>df_s = pd.DataFrame();<br \/>\ndata_file = &quot;per-day-fortune-30-company-stock-price-data.csv&quot;;<br \/>\ndf_s = pd.read_csv(&quot;.\/data\/&quot; + data_file, low_memory = False);<br \/>\ndf_s.head()<\/p>\n<h1>In[6]:<\/h1>\n<h1>You can see ANTM stock price data is empty<\/h1>\n<h1># Cure data such as replace missing\/null values, use correct data type, sort by date (not really required)<\/h1>\n<h1>In[7]:<\/h1>\n<h1>convert Date field to be a Date Type<\/h1>\n<p>df_s[&quot;Date&quot;] = df_s[&quot;Date&quot;].astype(&#8216;datetime64[ns]&#8217;)<\/p>\n<h1>Sort data by date although this is no longer needed as data already is sorted when I generated data<\/h1>\n<h1>df_s = df_s.sort_values( by = [&#39;Ticker&#39;,&#39;Date&#39;], ascending = True )<\/h1>\n<p>df_s = df_s.sort_values( by = &#8216;Date&#8217;, ascending = True )<br \/>\ndf_s.head()<\/p>\n<h1>In[8]:<\/h1>\n<h1><a href=\"https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.interpolate.html\">https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.interpolate.html<\/a><\/h1>\n<p>df_s_transpose = df_s<\/p>\n<p>try:<br \/>\ndf_s_transpose = df_s_transpose.interpolate(inplace = False)<br \/>\nexcept:<br \/>\nprint(&quot;An exception occurred. Operation ignored&quot;)<br \/>\nexit<\/p>\n<h1>check if any value is null<\/h1>\n<p>df_s_transpose.isnull().values.any()<\/p>\n<h1>check if any column (axis=1) is null<\/h1>\n<p>df_s_transpose[df_s_transpose.isna().any(axis = 1)]<\/p>\n<h1>In[9]:<\/h1>\n<p>df_s_transpose<\/p>\n<h1>In[10]:<\/h1>\n<h1>df_s_transpose = df_s<\/h1>\n<p>if drop_cols_with_na == 1:<br \/>\ndf_s_transpose = df_s_transpose.dropna(axis = 1);<\/p>\n<p>print(df_s_transpose.shape)<br \/>\ndf_s_transpose.head()<\/p>\n<h1>In[11]:<\/h1>\n<h1>further check and verify<\/h1>\n<p>df_s_transpose.isnull().values.any()<br \/>\ndf_s_transpose[df_s_transpose.isna().any( axis = 1 )]<\/p>\n<h1>In[12]:<\/h1>\n<h1>making the date column as the index column for the dataset<\/h1>\n<h1>df_s_transpose.index = df_s_transpose[&#39;Date&#39;]<\/h1>\n<p>df_s_transpose.index = df_s_transpose.index.astype(&#8216;datetime64[ns]&#8217;)<\/p>\n<h1># spearman Correlation Coefficient<\/h1>\n<h1>In[13]:<\/h1>\n<p>df_s_transpose_spearman = df_s_transpose.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\ndf_s_transpose_spearman<\/p>\n<h1># spearman Correlation Coefficient based Adjacency Graph Matrix<\/h1>\n<h1>In[14]:<\/h1>\n<p>df_s_transpose_spearman[df_s_transpose_spearman &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman[df_s_transpose_spearman &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman<\/p>\n<h1>In[15]:<\/h1>\n<h1>make the diagonal element to be zero. No self loop\/edge<\/h1>\n<p>import numpy as np<br \/>\nnp.fill_diagonal(df_s_transpose_spearman.values, 0)<br \/>\ndf_s_transpose_spearman<\/p>\n<h1>Create and visualize the Graphs<\/h1>\n<h1>In[17]:<\/h1>\n<p>import networkx as nx<br \/>\nGraph_spearman = nx.Graph(df_s_transpose_spearman)<\/p>\n<h1>In[18]:<\/h1>\n<p>nx.draw_networkx(Graph_spearman, pos = nx.circular_layout( Graph_spearman ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1># Experiment, we will divide the data into train, test, and validation graphs<\/h1>\n<h1>In[19]:<\/h1>\n<p>df_s_transpose.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\n#df_s_transpose[[{1,2,3}]]<br \/>\n#df_s_transpose.iloc[:, 0:10]<\/p>\n<h1>In[20]:<\/h1>\n<h1>Train Graph<\/h1>\n<h1>In[21]:<\/h1>\n<p>df_s_spearman_train = df_s_transpose.iloc[:, 0:15]<br \/>\ndf_s_transpose_spearman_train = df_s_spearman_train.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_train.values, 0)<\/p>\n<p>df_s_transpose_spearman_train[df_s_transpose_spearman_train &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_train[df_s_transpose_spearman_train &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_train<\/p>\n<p>df_s_transpose_spearman_train<\/p>\n<h1># Test Graph<\/h1>\n<h1>In[22]:<\/h1>\n<p>df_s_spearman_test = df_s_transpose.iloc[:, 15:] #df_s_transpose.iloc[:, 15:23]<br \/>\ndf_s_transpose_spearman_test = df_s_spearman_test.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_test.values, 0)<\/p>\n<p>df_s_transpose_spearman_train[df_s_transpose_spearman_test &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_train[df_s_transpose_spearman_test &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_test<\/p>\n<h1># Validation Graph<\/h1>\n<h1>In[23]:<\/h1>\n<p>df_s_spearman_validation = df_s_transpose.iloc[:, 15:] #df_s_transpose.iloc[:, 23:]<br \/>\ndf_s_transpose_spearman_validation = df_s_spearman_validation.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_validation.values, 0)<br \/>\ndf_s_transpose_spearman_validation<\/p>\n<p>df_s_transpose_spearman_validation[df_s_transpose_spearman_validation &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_validation[df_s_transpose_spearman_validation &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_validation<\/p>\n<h1>In[24]:<\/h1>\n<p>graph_spearman_train = nx.Graph(df_s_transpose_spearman_train)<br \/>\ngraph_spearman_test = nx.Graph(df_s_transpose_spearman_test)<br \/>\ngraph_spearman_validation = nx.Graph(df_s_transpose_spearman_validation)<\/p>\n<p>nx.draw_networkx(graph_spearman_train, pos = nx.circular_layout( graph_spearman_train ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1>In[25]:<\/h1>\n<p>df_s_spearman_train.corr(numeric_only = True)<\/p>\n<h1>In[26]:<\/h1>\n<p>nx.draw_networkx(graph_spearman_test, pos = nx.circular_layout( graph_spearman_test ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1>In[27]:<\/h1>\n<p>nx.draw_networkx(graph_spearman_validation, pos = nx.circular_layout( graph_spearman_validation ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1># Create GCN layer. spearman<\/h1>\n<h1># Find all stocks = nodes<\/h1>\n<h1>In[28]:<\/h1>\n<h1>improvement: make sure only stocks\/nodes that are in the graph are taken<\/h1>\n<p>all_stock_nodes = df_s_transpose_spearman.index.to_list()<br \/>\nall_stock_nodes[:5]<\/p>\n<h1># Find all edges between nodes<\/h1>\n<p>#<\/p>\n<h1>This may need adjustment to reflect train, test, validation graphs<\/h1>\n<h1>In[29]:<\/h1>\n<p>source = [];<br \/>\ntarget = [];<br \/>\nedge_feature = [];<\/p>\n<p>for aStock in all_stock_nodes:<br \/>\nfor anotherStock in all_stock_nodes:<br \/>\nif df_s_transpose_spearman[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\nsource.append(aStock)<br \/>\ntarget.append(anotherStock)<br \/>\nedge_feature.append(1)<\/p>\n<h1>edge feature is not required except for news based graph<\/h1>\n<p>source, target, edge_feature<\/p>\n<h1># Find all edges in Train, Test, and Validation Graphs<\/h1>\n<h1>In[30]:<\/h1>\n<p>trainSource = [];<br \/>\ntrainTarget = [];<br \/>\ntrainEdge_feature = [];<br \/>\ntrainNodeList = df_s_transpose_spearman_train.index.to_list();<\/p>\n<p>testSource = [];<br \/>\ntestTarget = [];<br \/>\ntestEdge_feature = [];<br \/>\ntestNodeList = df_s_transpose_spearman_test.index.to_list();<\/p>\n<p>validationSource = [];<br \/>\nvalidationTarget = [];<br \/>\nvalidationEdge_feature = [];<br \/>\nvalidationNodeList = df_s_transpose_spearman_validation.index.to_list();<\/p>\n<p>for aStock in trainNodeList:<br \/>\nfor anotherStock in trainNodeList:<br \/>\nif df_s_transpose_spearman_train[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\ntrainSource.append(aStock)<br \/>\ntrainTarget.append(anotherStock)<br \/>\ntrainEdge_feature.append(1)<\/p>\n<p>for aStock in testNodeList:<br \/>\nfor anotherStock in testNodeList:<br \/>\nif df_s_transpose_spearman_test[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\ntestSource.append(aStock)<br \/>\ntestTarget.append(anotherStock)<br \/>\ntestEdge_feature.append(1)<\/p>\n<p>for aStock in validationNodeList:<br \/>\nfor anotherStock in validationNodeList:<br \/>\nif df_s_transpose_spearman_validation[aStock][anotherStock] &gt; 0:<\/p>\n<h1>print(df_s_transpose_spearman[aStock][anotherStock])<\/h1>\n<p>validationSource.append(aStock)<br \/>\nvalidationTarget.append(anotherStock)<br \/>\nvalidationEdge_feature.append(1)<\/p>\n<h1>edge feature is not required except for news based graph<\/h1>\n<p>trainSource, trainTarget, trainEdge_feature<br \/>\ntestSource, testTarget, testEdge_feature<br \/>\nvalidationSource, validationTarget, validationEdge_feature<\/p>\n<h1># Create variables to create stellar graph<\/h1>\n<h1># Edges<\/h1>\n<h1>In[31]:<\/h1>\n<h1><a href=\"https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html\">https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html<\/a><\/h1>\n<p>spearman_edges = pd.DataFrame(<br \/>\n{&quot;source&quot;: source, &quot;target&quot;: target}<br \/>\n)<\/p>\n<p>spearman_edges_data = pd.DataFrame(<br \/>\n{&quot;source&quot;: source, &quot;target&quot;: target, &quot;edge_feature&quot;: edge_feature}<br \/>\n)<\/p>\n<h1><a href=\"https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html\">https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html<\/a><\/h1>\n<p>spearman_edges_train = pd.DataFrame(<br \/>\n{&quot;source&quot;: trainSource, &quot;target&quot;: trainTarget}<br \/>\n)<\/p>\n<p>spearman_edges_data_train = pd.DataFrame(<br \/>\n{&quot;source&quot;: trainSource, &quot;target&quot;: trainTarget, &quot;edge_feature&quot;: trainEdge_feature}<br \/>\n)<\/p>\n<p>spearman_edges_test = pd.DataFrame(<br \/>\n{&quot;source&quot;: testSource, &quot;target&quot;: testTarget}<br \/>\n)<\/p>\n<p>spearman_edges_data_test = pd.DataFrame(<br \/>\n{&quot;source&quot;: testSource, &quot;target&quot;: testTarget, &quot;edge_feature&quot;: testEdge_feature}<br \/>\n)<\/p>\n<p>spearman_edges_validation = pd.DataFrame(<br \/>\n{&quot;source&quot;: validationSource, &quot;target&quot;: validationTarget}<br \/>\n)<\/p>\n<p>spearman_edges_train[:10]<\/p>\n<h1># Have the time series data as part of the nodes<\/h1>\n<h1># Structure the Feature Matrix so that it can be passed to the GCN<\/h1>\n<h1>In[32]:<\/h1>\n<p>df_s_transpose_feature = df_s_transpose.reset_index(drop = True, inplace = False)<\/p>\n<h1>df_s_transpose_feature = df_s_transpose_feature.values.tolist()<\/h1>\n<h1>print(df_s_transpose_feature.values.tolist())<\/h1>\n<p>#df_s_transpose_feature[&#39;WY&#39;].values<br \/>\ndf_s_transpose_feature[&#39;AAPL&#39;].shape, df_s_transpose_feature[&#39;AAPL&#39;].values<\/p>\n<h1>In[33]:<\/h1>\n<p>len(all_stock_nodes)<\/p>\n<h1>In[34]:<\/h1>\n<h1>bring\/assign data to nodes<\/h1>\n<p>node_Data = [];<br \/>\nfor x in all_stock_nodes:<br \/>\nnode_Data.append( df_s_transpose_feature[x].values)<\/p>\n<p>node_Data<\/p>\n<h1>In[35]:<\/h1>\n<h1>convert node data variable into a dataframe so that the data structure is compatible with graph NN<\/h1>\n<p>spearman_graph_node_data = pd.DataFrame(node_Data, index = all_stock_nodes)<br \/>\nspearman_graph_node_data.head()<\/p>\n<h1>In[36]:<\/h1>\n<p>node_Data[14:15],<br \/>\nlen(validationNodeList)<br \/>\nlen(testNodeList)<\/p>\n<h1>In[37]:<\/h1>\n<h1>Node time series data based on train, test, validation graph<\/h1>\n<h1>In[38]:<\/h1>\n<h1>convert node data variable into a dataframe so that the data structure is compatible with graph NN<\/h1>\n<p>spearman_graph_node_data_train = pd.DataFrame(node_Data[0:14], index = trainNodeList)<br \/>\nspearman_graph_node_data_train.head()<\/p>\n<p>spearman_graph_node_data_test = pd.DataFrame(node_Data[14:], index = testNodeList) #pd.DataFrame(node_Data[15:23], index = testNodeList)<br \/>\nspearman_graph_node_data_test.head()<\/p>\n<p>spearman_graph_node_data_validation = pd.DataFrame(node_Data[14:], index = validationNodeList) #pd.DataFrame(node_Data[22:30], index = validationNodeList)<br \/>\nspearman_graph_node_data_validation.head()<\/p>\n<h1>In[39]:<\/h1>\n<p>spearman_graph_node_data_train<\/p>\n<h1># Graph (stellar) with features as part of Nodes<\/h1>\n<h1>In[40]:<\/h1>\n<h1>Overall<\/h1>\n<p>spearman_graph_with_node_features = StellarGraph(spearman_graph_node_data, edges = spearman_edges, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_graph_with_node_features.info<\/a>())<\/p>\n<h1>train nodes<\/h1>\n<p>spearman_train_graph_with_node_features = StellarGraph(spearman_graph_node_data_train, edges = spearman_edges_train, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_train_graph_with_node_features.info<\/a>())<\/p>\n<h1>test<\/h1>\n<p>spearman_test_graph_with_node_features = StellarGraph(spearman_graph_node_data_test, edges = spearman_edges_test, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_test_graph_with_node_features.info<\/a>())<\/p>\n<h1>validation<\/h1>\n<p>spearman_validation_graph_with_node_features = StellarGraph(spearman_graph_node_data_validation, edges = spearman_edges_validation, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_validation_graph_with_node_features.info<\/a>())<\/p>\n<h1># Adapting everything for DeepGraphCNN<\/h1>\n<h1>In[41]:<\/h1>\n<p>spearman_graph_node_data.iloc[0:15, :]<\/p>\n<h1># Graphs to be jused for DeepGraphCNN<\/h1>\n<h1>In[42]:<\/h1>\n<p>graphs = list()<br \/>\n#graphs.append(spearman_graph_with_node_features)<br \/>\ngraphs.append(spearman_train_graph_with_node_features)<br \/>\ngraphs.append(spearman_test_graph_with_node_features)<br \/>\ngraphs.append(spearman_validation_graph_with_node_features)<\/p>\n<h1>In[43]:<\/h1>\n<p>summary = pd.DataFrame(<br \/>\n[(g.number_of_nodes(), g.number_of_edges()) for g in graphs],<br \/>\ncolumns=[&quot;nodes&quot;, &quot;edges&quot;],<br \/>\n)<br \/>\nsummary.describe().round()<\/p>\n<h1>In[44]:<\/h1>\n<h1>graph_labels = all_stock_nodes<\/h1>\n<h1>In[45]:<\/h1>\n<h1>Generator<\/h1>\n<p>#generator = FullBatchNodeGenerator(spearman_graph_with_node_features, method = &quot;gcn&quot;) # , sparse = False<br \/>\n#vars(generator)<\/p>\n<p>generator = PaddedGraphGenerator( graphs = graphs)<\/p>\n<h1>generator = PaddedGraphGenerator( spearman_graph_with_node_features)<\/h1>\n<h1>In[46]:<\/h1>\n<p>vars(generator)<\/p>\n<h1># Train Test Split<\/h1>\n<h1># Commented out on 2023-04-18<\/h1>\n<h1>train_subjects, test_subjects = model_selection.train_test_split(<\/h1>\n<h1>spearman_graph_node_data<\/h1>\n<h1>)<\/h1>\n<p>#<\/p>\n<h1>val_subjects, test_subjects_step_2 = model_selection.train_test_split(<\/h1>\n<h1>test_subjects<\/h1>\n<h1>)<\/h1>\n<p>#<\/p>\n<h1>#, train_size = 500, test_size = None, stratify = test_subjects<\/h1>\n<p>#<\/p>\n<h1>train_subjects.shape, test_subjects.shape, val_subjects.shape, test_subjects_step_2.shape<\/h1>\n<h1>In[98]:<\/h1>\n","protected":false},"excerpt":{"rendered":"<p>#!\/usr\/bin\/env python coding: utf-8 # 3rd Model: Deepgraph CNN: Stock Price Prediction using DeepGraphCNN Neural Networks. It includes GCN layers and CNN layers. I have added an MLP at the last layer to predict stock prices. # # Input graphs were created for spearman, Spearman, and Kendal Tau correlations\/coefficients from historical stock prices. Also, another &hellip; <\/p>\n<p><a class=\"more-link btn\" href=\"http:\/\/bangla.sitestree.com\/?p=76091\">Continue reading<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[182],"tags":[],"class_list":["post-76091","post","type-post","status-publish","format-standard","hentry","category---blog","item-wrap"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":78242,"url":"http:\/\/bangla.sitestree.com\/?p=78242","url_meta":{"origin":76091,"position":0},"title":"Statistics for Data Analytics and Machine Learning Projects","author":"Sayed","date":"May 22, 2025","format":false,"excerpt":"\u2022Null Hypothesis \u2022[2] \u2022Paired t-test \u2022Unpaired t-test \u2022Pearson Correlation \u2022One Way: Analysis of variance \u2022Spearman Correlation \u2022Spearman \u2022Kendal Tau Coef \u2022Wilcoxon Sum test \u2022Basic EDA \u2022Mcnaimer\u2019s test \u2022Friedman test \u2022Kruskal-Wallis Test \u2022Two Way Analysis of variance \u2022K-Fold Cross Validation paired t-test \u2022Wilcoxon Signed Rank Test Data Analytics, Machine Learning, Data\u2026","rel":"","context":"In &quot;Analytics and Machine Learning Project Development&quot;","block_context":{"text":"Analytics and Machine Learning Project Development","link":"http:\/\/bangla.sitestree.com\/?cat=1974"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=525%2C300 1.5x"},"classes":[]},{"id":78291,"url":"http:\/\/bangla.sitestree.com\/?p=78291","url_meta":{"origin":76091,"position":1},"title":"Pearson vs Spearman Correlation","author":"Sayed","date":"June 1, 2025","format":false,"excerpt":"Pearson: Generally Linear relation Assumes Linearity. Correlation between height and weight Sensitive to outliers. Spearman: increasing or decreasing relationship, but may not be linear. Monotonic. Higher marks lead to lower ranks, but generally not linearly. Does not assume Linearity. It can be good for categorical variables and relationships. Less Sensitive\u2026","rel":"","context":"In &quot;Analytics and Machine Learning Project Development&quot;","block_context":{"text":"Analytics and Machine Learning Project Development","link":"http:\/\/bangla.sitestree.com\/?cat=1974"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":14811,"url":"http:\/\/bangla.sitestree.com\/?p=14811","url_meta":{"origin":76091,"position":2},"title":"Next Recession: How to Invest and Profit in the Next Recession","author":"Sayed","date":"June 17, 2019","format":false,"excerpt":"How to Invest and Profit in the Next Recession https:\/\/www.bloomberg.com\/opinion\/articles\/2019-06-17\/how-to-invest-and-profit-in-the-next-recession \" plan on deploying your cash in tranches: Buy a U.S. index fund when markets are down 20 to 25%; add a developed global index fund when markets fall by 30%. And if we are lucky enough to enjoy a\u2026","rel":"","context":"In &quot;AI ML DS RL DL NN NLP Data Mining Optimization&quot;","block_context":{"text":"AI ML DS RL DL NN NLP Data Mining Optimization","link":"http:\/\/bangla.sitestree.com\/?cat=1910"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":25039,"url":"http:\/\/bangla.sitestree.com\/?p=25039","url_meta":{"origin":76091,"position":3},"title":"Next Recession: How to Invest and Profit in the Next Recession #Root","author":"Author-Check- Article-or-Video","date":"April 15, 2021","format":false,"excerpt":"How to Invest and Profit in the Next Recession https:\/\/www.bloomberg.com\/opinion\/articles\/2019-06-17\/how-to-invest-and-profit-in-the-next-recession \" plan on deploying your cash in tranches: Buy a U.S. index fund when markets are down 20 to 25%; add a developed global index fund when markets fall by 30%. And if we are lucky enough to enjoy a\u2026","rel":"","context":"In &quot;FromSitesTree.com&quot;","block_context":{"text":"FromSitesTree.com","link":"http:\/\/bangla.sitestree.com\/?cat=1917"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":76436,"url":"http:\/\/bangla.sitestree.com\/?p=76436","url_meta":{"origin":76091,"position":4},"title":"1. Libraries used for the project: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)","author":"Sayed","date":"December 4, 2024","format":false,"excerpt":"#import libraries import osimport pandas as pdimport math #Import Libraries for Graph, GNN, and GCN import stellargraph as sgfrom stellargraph import StellarGraphfrom stellargraph.layer import DeepGraphCNNfrom stellargraph.mapper import FullBatchNodeGeneratorfrom stellargraph.mapper import PaddedGraphGeneratorfrom stellargraph.layer import GCN #Machine Learnig related library Imports from tensorflow.keras import layers, optimizers, losses, metrics, Modelfrom sklearn import preprocessing,\u2026","rel":"","context":"In &quot;Code: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)&quot;","block_context":{"text":"Code: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)","link":"http:\/\/bangla.sitestree.com\/?cat=1969"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16302,"url":"http:\/\/bangla.sitestree.com\/?p=16302","url_meta":{"origin":76091,"position":5},"title":"Graph Mining: Shared Nearest Neighbors : Clustering : Community Detection","author":"Sayed","date":"October 7, 2019","format":false,"excerpt":"Graph Mining: Shared Nearest Neighbors : Clustering : Community Detection Graph Mining: Shared Nearest Neighbors (SNN): Clustering : Community Detection: Learn by Finding Answers to the Following Questions. Will use SNN sometimes. What is one another name of the algorithm: Shared Nearest Neighbors? What is the purpose of the Algorithm:\u2026","rel":"","context":"In &quot;Graph Mining&quot;","block_context":{"text":"Graph Mining","link":"http:\/\/bangla.sitestree.com\/?cat=1905"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]}],"_links":{"self":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/76091","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=76091"}],"version-history":[{"count":0,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/76091\/revisions"}],"wp:attachment":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=76091"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=76091"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=76091"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}