{"id":76091,"date":"2024-05-19T23:01:21","date_gmt":"2024-05-20T03:01:21","guid":{"rendered":"https:\/\/bangla.sitestree.com\/spearman-correlation-coefficient-and-graph-mining\/"},"modified":"2024-05-19T23:01:21","modified_gmt":"2024-05-20T03:01:21","slug":"spearman-correlation-coefficient-and-graph-mining","status":"publish","type":"post","link":"http:\/\/bangla.sitestree.com\/?p=76091","title":{"rendered":"Spearman Correlation Coefficient and Graph Mining"},"content":{"rendered":"<p>#!\/usr\/bin\/env python<\/p>\n<h1>coding: utf-8<\/h1>\n<h1># 3rd Model: Deepgraph CNN: Stock Price Prediction using DeepGraphCNN Neural Networks. It includes GCN layers and CNN layers. I have added an MLP at the last layer to predict stock prices.<\/h1>\n<p>#<\/p>\n<h1># Input graphs were created for spearman, Spearman, and Kendal Tau correlations\/coefficients from historical stock prices. Also, another graph is created based on financial news articles.<\/h1>\n<p>#<\/p>\n<h1># For the sake of making execution easier (and at once), I have kept multiple approaches (spearman, Spearman, and Kendal Tau, News Based) in the same file. One big code file can be difficult to handle; is done just for making execution easier.<\/h1>\n<p>#<\/p>\n<h1># Because I initially tried separately and brought the code together, some code might be a bit redundant\/repeating. I may have done some cleaning.<\/h1>\n<p>#<\/p>\n<h1># An use case of DeepGraphCNN for Node Classification<\/h1>\n<h1># <a href=\"https:\/\/stellargraph.readthedocs.io\/en\/latest\/demos\/graph-classification\/dgcnn-graph-classification.html\">https:\/\/stellargraph.readthedocs.io\/en\/latest\/demos\/graph-classification\/dgcnn-graph-classification.html<\/a><\/h1>\n<p>#<\/p>\n<h1># Import Libraries<\/h1>\n<h1>In[1]:<\/h1>\n<h1>import libraries<\/h1>\n<p>import os<br \/>\nimport pandas as pd<br \/>\nimport math<\/p>\n<h1>In[2]:<\/h1>\n<h1>Import Libraries for Graph, GNN, and GCN<\/h1>\n<p>import stellargraph as sg<br \/>\nfrom stellargraph import StellarGraph<br \/>\nfrom stellargraph.layer import DeepGraphCNN<br \/>\nfrom stellargraph.mapper import FullBatchNodeGenerator<br \/>\nfrom stellargraph.mapper import PaddedGraphGenerator<br \/>\nfrom stellargraph.layer import GCN<\/p>\n<h1>In[3]:<\/h1>\n<h1>Machine Learnig related library Imports<\/h1>\n<p>from tensorflow.keras import layers, optimizers, losses, metrics, Model<br \/>\nfrom sklearn import preprocessing, model_selection<br \/>\nfrom IPython.display import display, HTML<br \/>\nimport matplotlib.pyplot as plt<br \/>\nget_ipython().run_line_magic(&#8216;matplotlib&#8217;, &#8216;inline&#8217;)<br \/>\nfrom tensorflow.keras.layers import Dense, Conv1D, MaxPool1D, Dropout, Flatten<br \/>\nfrom tensorflow import keras<\/p>\n<h1>In[4]:<\/h1>\n<h1>If we want to drop NAN column or row wise for stock price data<\/h1>\n<h1>I did not need to use this options that much<\/h1>\n<p>drop_cols_with_na = 1<br \/>\ndrop_rows_with_na = 1<\/p>\n<h1># Dataset: Using 30 companies from the Fortune 500 companies (the paper used these stocks)<\/h1>\n<h1>In[5]:<\/h1>\n<p>df_s = pd.DataFrame();<br \/>\ndata_file = &quot;per-day-fortune-30-company-stock-price-data.csv&quot;;<br \/>\ndf_s = pd.read_csv(&quot;.\/data\/&quot; + data_file, low_memory = False);<br \/>\ndf_s.head()<\/p>\n<h1>In[6]:<\/h1>\n<h1>You can see ANTM stock price data is empty<\/h1>\n<h1># Cure data such as replace missing\/null values, use correct data type, sort by date (not really required)<\/h1>\n<h1>In[7]:<\/h1>\n<h1>convert Date field to be a Date Type<\/h1>\n<p>df_s[&quot;Date&quot;] = df_s[&quot;Date&quot;].astype(&#8216;datetime64[ns]&#8217;)<\/p>\n<h1>Sort data by date although this is no longer needed as data already is sorted when I generated data<\/h1>\n<h1>df_s = df_s.sort_values( by = [&#39;Ticker&#39;,&#39;Date&#39;], ascending = True )<\/h1>\n<p>df_s = df_s.sort_values( by = &#8216;Date&#8217;, ascending = True )<br \/>\ndf_s.head()<\/p>\n<h1>In[8]:<\/h1>\n<h1><a href=\"https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.interpolate.html\">https:\/\/pandas.pydata.org\/pandas-docs\/stable\/reference\/api\/pandas.DataFrame.interpolate.html<\/a><\/h1>\n<p>df_s_transpose = df_s<\/p>\n<p>try:<br \/>\ndf_s_transpose = df_s_transpose.interpolate(inplace = False)<br \/>\nexcept:<br \/>\nprint(&quot;An exception occurred. Operation ignored&quot;)<br \/>\nexit<\/p>\n<h1>check if any value is null<\/h1>\n<p>df_s_transpose.isnull().values.any()<\/p>\n<h1>check if any column (axis=1) is null<\/h1>\n<p>df_s_transpose[df_s_transpose.isna().any(axis = 1)]<\/p>\n<h1>In[9]:<\/h1>\n<p>df_s_transpose<\/p>\n<h1>In[10]:<\/h1>\n<h1>df_s_transpose = df_s<\/h1>\n<p>if drop_cols_with_na == 1:<br \/>\ndf_s_transpose = df_s_transpose.dropna(axis = 1);<\/p>\n<p>print(df_s_transpose.shape)<br \/>\ndf_s_transpose.head()<\/p>\n<h1>In[11]:<\/h1>\n<h1>further check and verify<\/h1>\n<p>df_s_transpose.isnull().values.any()<br \/>\ndf_s_transpose[df_s_transpose.isna().any( axis = 1 )]<\/p>\n<h1>In[12]:<\/h1>\n<h1>making the date column as the index column for the dataset<\/h1>\n<h1>df_s_transpose.index = df_s_transpose[&#39;Date&#39;]<\/h1>\n<p>df_s_transpose.index = df_s_transpose.index.astype(&#8216;datetime64[ns]&#8217;)<\/p>\n<h1># spearman Correlation Coefficient<\/h1>\n<h1>In[13]:<\/h1>\n<p>df_s_transpose_spearman = df_s_transpose.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\ndf_s_transpose_spearman<\/p>\n<h1># spearman Correlation Coefficient based Adjacency Graph Matrix<\/h1>\n<h1>In[14]:<\/h1>\n<p>df_s_transpose_spearman[df_s_transpose_spearman &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman[df_s_transpose_spearman &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman<\/p>\n<h1>In[15]:<\/h1>\n<h1>make the diagonal element to be zero. No self loop\/edge<\/h1>\n<p>import numpy as np<br \/>\nnp.fill_diagonal(df_s_transpose_spearman.values, 0)<br \/>\ndf_s_transpose_spearman<\/p>\n<h1>Create and visualize the Graphs<\/h1>\n<h1>In[17]:<\/h1>\n<p>import networkx as nx<br \/>\nGraph_spearman = nx.Graph(df_s_transpose_spearman)<\/p>\n<h1>In[18]:<\/h1>\n<p>nx.draw_networkx(Graph_spearman, pos = nx.circular_layout( Graph_spearman ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1># Experiment, we will divide the data into train, test, and validation graphs<\/h1>\n<h1>In[19]:<\/h1>\n<p>df_s_transpose.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\n#df_s_transpose[[{1,2,3}]]<br \/>\n#df_s_transpose.iloc[:, 0:10]<\/p>\n<h1>In[20]:<\/h1>\n<h1>Train Graph<\/h1>\n<h1>In[21]:<\/h1>\n<p>df_s_spearman_train = df_s_transpose.iloc[:, 0:15]<br \/>\ndf_s_transpose_spearman_train = df_s_spearman_train.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_train.values, 0)<\/p>\n<p>df_s_transpose_spearman_train[df_s_transpose_spearman_train &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_train[df_s_transpose_spearman_train &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_train<\/p>\n<p>df_s_transpose_spearman_train<\/p>\n<h1># Test Graph<\/h1>\n<h1>In[22]:<\/h1>\n<p>df_s_spearman_test = df_s_transpose.iloc[:, 15:] #df_s_transpose.iloc[:, 15:23]<br \/>\ndf_s_transpose_spearman_test = df_s_spearman_test.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_test.values, 0)<\/p>\n<p>df_s_transpose_spearman_train[df_s_transpose_spearman_test &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_train[df_s_transpose_spearman_test &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_test<\/p>\n<h1># Validation Graph<\/h1>\n<h1>In[23]:<\/h1>\n<p>df_s_spearman_validation = df_s_transpose.iloc[:, 15:] #df_s_transpose.iloc[:, 23:]<br \/>\ndf_s_transpose_spearman_validation = df_s_spearman_validation.corr(method = &#8216;spearman&#8217;, numeric_only = True)<br \/>\nnp.fill_diagonal(df_s_transpose_spearman_validation.values, 0)<br \/>\ndf_s_transpose_spearman_validation<\/p>\n<p>df_s_transpose_spearman_validation[df_s_transpose_spearman_validation &gt;= 0.4] = 1<br \/>\ndf_s_transpose_spearman_validation[df_s_transpose_spearman_validation &lt; 0.4] = 0<br \/>\ndf_s_transpose_spearman_validation<\/p>\n<h1>In[24]:<\/h1>\n<p>graph_spearman_train = nx.Graph(df_s_transpose_spearman_train)<br \/>\ngraph_spearman_test = nx.Graph(df_s_transpose_spearman_test)<br \/>\ngraph_spearman_validation = nx.Graph(df_s_transpose_spearman_validation)<\/p>\n<p>nx.draw_networkx(graph_spearman_train, pos = nx.circular_layout( graph_spearman_train ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1>In[25]:<\/h1>\n<p>df_s_spearman_train.corr(numeric_only = True)<\/p>\n<h1>In[26]:<\/h1>\n<p>nx.draw_networkx(graph_spearman_test, pos = nx.circular_layout( graph_spearman_test ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1>In[27]:<\/h1>\n<p>nx.draw_networkx(graph_spearman_validation, pos = nx.circular_layout( graph_spearman_validation ), node_color = &#8216;r&#8217;, edge_color = &#8216;b&#8217;)<\/p>\n<h1># Create GCN layer. spearman<\/h1>\n<h1># Find all stocks = nodes<\/h1>\n<h1>In[28]:<\/h1>\n<h1>improvement: make sure only stocks\/nodes that are in the graph are taken<\/h1>\n<p>all_stock_nodes = df_s_transpose_spearman.index.to_list()<br \/>\nall_stock_nodes[:5]<\/p>\n<h1># Find all edges between nodes<\/h1>\n<p>#<\/p>\n<h1>This may need adjustment to reflect train, test, validation graphs<\/h1>\n<h1>In[29]:<\/h1>\n<p>source = [];<br \/>\ntarget = [];<br \/>\nedge_feature = [];<\/p>\n<p>for aStock in all_stock_nodes:<br \/>\nfor anotherStock in all_stock_nodes:<br \/>\nif df_s_transpose_spearman[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\nsource.append(aStock)<br \/>\ntarget.append(anotherStock)<br \/>\nedge_feature.append(1)<\/p>\n<h1>edge feature is not required except for news based graph<\/h1>\n<p>source, target, edge_feature<\/p>\n<h1># Find all edges in Train, Test, and Validation Graphs<\/h1>\n<h1>In[30]:<\/h1>\n<p>trainSource = [];<br \/>\ntrainTarget = [];<br \/>\ntrainEdge_feature = [];<br \/>\ntrainNodeList = df_s_transpose_spearman_train.index.to_list();<\/p>\n<p>testSource = [];<br \/>\ntestTarget = [];<br \/>\ntestEdge_feature = [];<br \/>\ntestNodeList = df_s_transpose_spearman_test.index.to_list();<\/p>\n<p>validationSource = [];<br \/>\nvalidationTarget = [];<br \/>\nvalidationEdge_feature = [];<br \/>\nvalidationNodeList = df_s_transpose_spearman_validation.index.to_list();<\/p>\n<p>for aStock in trainNodeList:<br \/>\nfor anotherStock in trainNodeList:<br \/>\nif df_s_transpose_spearman_train[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\ntrainSource.append(aStock)<br \/>\ntrainTarget.append(anotherStock)<br \/>\ntrainEdge_feature.append(1)<\/p>\n<p>for aStock in testNodeList:<br \/>\nfor anotherStock in testNodeList:<br \/>\nif df_s_transpose_spearman_test[aStock][anotherStock] &gt; 0:<br \/>\n#print(df_s_transpose_spearman[aStock][anotherStock])<br \/>\ntestSource.append(aStock)<br \/>\ntestTarget.append(anotherStock)<br \/>\ntestEdge_feature.append(1)<\/p>\n<p>for aStock in validationNodeList:<br \/>\nfor anotherStock in validationNodeList:<br \/>\nif df_s_transpose_spearman_validation[aStock][anotherStock] &gt; 0:<\/p>\n<h1>print(df_s_transpose_spearman[aStock][anotherStock])<\/h1>\n<p>validationSource.append(aStock)<br \/>\nvalidationTarget.append(anotherStock)<br \/>\nvalidationEdge_feature.append(1)<\/p>\n<h1>edge feature is not required except for news based graph<\/h1>\n<p>trainSource, trainTarget, trainEdge_feature<br \/>\ntestSource, testTarget, testEdge_feature<br \/>\nvalidationSource, validationTarget, validationEdge_feature<\/p>\n<h1># Create variables to create stellar graph<\/h1>\n<h1># Edges<\/h1>\n<h1>In[31]:<\/h1>\n<h1><a href=\"https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html\">https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html<\/a><\/h1>\n<p>spearman_edges = pd.DataFrame(<br \/>\n{&quot;source&quot;: source, &quot;target&quot;: target}<br \/>\n)<\/p>\n<p>spearman_edges_data = pd.DataFrame(<br \/>\n{&quot;source&quot;: source, &quot;target&quot;: target, &quot;edge_feature&quot;: edge_feature}<br \/>\n)<\/p>\n<h1><a href=\"https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html\">https:\/\/stellargraph.readthedocs.io\/en\/stable\/demos\/basics\/loading-pandas.html<\/a><\/h1>\n<p>spearman_edges_train = pd.DataFrame(<br \/>\n{&quot;source&quot;: trainSource, &quot;target&quot;: trainTarget}<br \/>\n)<\/p>\n<p>spearman_edges_data_train = pd.DataFrame(<br \/>\n{&quot;source&quot;: trainSource, &quot;target&quot;: trainTarget, &quot;edge_feature&quot;: trainEdge_feature}<br \/>\n)<\/p>\n<p>spearman_edges_test = pd.DataFrame(<br \/>\n{&quot;source&quot;: testSource, &quot;target&quot;: testTarget}<br \/>\n)<\/p>\n<p>spearman_edges_data_test = pd.DataFrame(<br \/>\n{&quot;source&quot;: testSource, &quot;target&quot;: testTarget, &quot;edge_feature&quot;: testEdge_feature}<br \/>\n)<\/p>\n<p>spearman_edges_validation = pd.DataFrame(<br \/>\n{&quot;source&quot;: validationSource, &quot;target&quot;: validationTarget}<br \/>\n)<\/p>\n<p>spearman_edges_train[:10]<\/p>\n<h1># Have the time series data as part of the nodes<\/h1>\n<h1># Structure the Feature Matrix so that it can be passed to the GCN<\/h1>\n<h1>In[32]:<\/h1>\n<p>df_s_transpose_feature = df_s_transpose.reset_index(drop = True, inplace = False)<\/p>\n<h1>df_s_transpose_feature = df_s_transpose_feature.values.tolist()<\/h1>\n<h1>print(df_s_transpose_feature.values.tolist())<\/h1>\n<p>#df_s_transpose_feature[&#39;WY&#39;].values<br \/>\ndf_s_transpose_feature[&#39;AAPL&#39;].shape, df_s_transpose_feature[&#39;AAPL&#39;].values<\/p>\n<h1>In[33]:<\/h1>\n<p>len(all_stock_nodes)<\/p>\n<h1>In[34]:<\/h1>\n<h1>bring\/assign data to nodes<\/h1>\n<p>node_Data = [];<br \/>\nfor x in all_stock_nodes:<br \/>\nnode_Data.append( df_s_transpose_feature[x].values)<\/p>\n<p>node_Data<\/p>\n<h1>In[35]:<\/h1>\n<h1>convert node data variable into a dataframe so that the data structure is compatible with graph NN<\/h1>\n<p>spearman_graph_node_data = pd.DataFrame(node_Data, index = all_stock_nodes)<br \/>\nspearman_graph_node_data.head()<\/p>\n<h1>In[36]:<\/h1>\n<p>node_Data[14:15],<br \/>\nlen(validationNodeList)<br \/>\nlen(testNodeList)<\/p>\n<h1>In[37]:<\/h1>\n<h1>Node time series data based on train, test, validation graph<\/h1>\n<h1>In[38]:<\/h1>\n<h1>convert node data variable into a dataframe so that the data structure is compatible with graph NN<\/h1>\n<p>spearman_graph_node_data_train = pd.DataFrame(node_Data[0:14], index = trainNodeList)<br \/>\nspearman_graph_node_data_train.head()<\/p>\n<p>spearman_graph_node_data_test = pd.DataFrame(node_Data[14:], index = testNodeList) #pd.DataFrame(node_Data[15:23], index = testNodeList)<br \/>\nspearman_graph_node_data_test.head()<\/p>\n<p>spearman_graph_node_data_validation = pd.DataFrame(node_Data[14:], index = validationNodeList) #pd.DataFrame(node_Data[22:30], index = validationNodeList)<br \/>\nspearman_graph_node_data_validation.head()<\/p>\n<h1>In[39]:<\/h1>\n<p>spearman_graph_node_data_train<\/p>\n<h1># Graph (stellar) with features as part of Nodes<\/h1>\n<h1>In[40]:<\/h1>\n<h1>Overall<\/h1>\n<p>spearman_graph_with_node_features = StellarGraph(spearman_graph_node_data, edges = spearman_edges, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_graph_with_node_features.info<\/a>())<\/p>\n<h1>train nodes<\/h1>\n<p>spearman_train_graph_with_node_features = StellarGraph(spearman_graph_node_data_train, edges = spearman_edges_train, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_train_graph_with_node_features.info<\/a>())<\/p>\n<h1>test<\/h1>\n<p>spearman_test_graph_with_node_features = StellarGraph(spearman_graph_node_data_test, edges = spearman_edges_test, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_test_graph_with_node_features.info<\/a>())<\/p>\n<h1>validation<\/h1>\n<p>spearman_validation_graph_with_node_features = StellarGraph(spearman_graph_node_data_validation, edges = spearman_edges_validation, node_type_default = &quot;corner&quot;, edge_type_default = &quot;line&quot;)<br \/>\nprint(<a href=\"\">spearman_validation_graph_with_node_features.info<\/a>())<\/p>\n<h1># Adapting everything for DeepGraphCNN<\/h1>\n<h1>In[41]:<\/h1>\n<p>spearman_graph_node_data.iloc[0:15, :]<\/p>\n<h1># Graphs to be jused for DeepGraphCNN<\/h1>\n<h1>In[42]:<\/h1>\n<p>graphs = list()<br \/>\n#graphs.append(spearman_graph_with_node_features)<br \/>\ngraphs.append(spearman_train_graph_with_node_features)<br \/>\ngraphs.append(spearman_test_graph_with_node_features)<br \/>\ngraphs.append(spearman_validation_graph_with_node_features)<\/p>\n<h1>In[43]:<\/h1>\n<p>summary = pd.DataFrame(<br \/>\n[(g.number_of_nodes(), g.number_of_edges()) for g in graphs],<br \/>\ncolumns=[&quot;nodes&quot;, &quot;edges&quot;],<br \/>\n)<br \/>\nsummary.describe().round()<\/p>\n<h1>In[44]:<\/h1>\n<h1>graph_labels = all_stock_nodes<\/h1>\n<h1>In[45]:<\/h1>\n<h1>Generator<\/h1>\n<p>#generator = FullBatchNodeGenerator(spearman_graph_with_node_features, method = &quot;gcn&quot;) # , sparse = False<br \/>\n#vars(generator)<\/p>\n<p>generator = PaddedGraphGenerator( graphs = graphs)<\/p>\n<h1>generator = PaddedGraphGenerator( spearman_graph_with_node_features)<\/h1>\n<h1>In[46]:<\/h1>\n<p>vars(generator)<\/p>\n<h1># Train Test Split<\/h1>\n<h1># Commented out on 2023-04-18<\/h1>\n<h1>train_subjects, test_subjects = model_selection.train_test_split(<\/h1>\n<h1>spearman_graph_node_data<\/h1>\n<h1>)<\/h1>\n<p>#<\/p>\n<h1>val_subjects, test_subjects_step_2 = model_selection.train_test_split(<\/h1>\n<h1>test_subjects<\/h1>\n<h1>)<\/h1>\n<p>#<\/p>\n<h1>#, train_size = 500, test_size = None, stratify = test_subjects<\/h1>\n<p>#<\/p>\n<h1>train_subjects.shape, test_subjects.shape, val_subjects.shape, test_subjects_step_2.shape<\/h1>\n<h1>In[98]:<\/h1>\n","protected":false},"excerpt":{"rendered":"<p>#!\/usr\/bin\/env python coding: utf-8 # 3rd Model: Deepgraph CNN: Stock Price Prediction using DeepGraphCNN Neural Networks. It includes GCN layers and CNN layers. I have added an MLP at the last layer to predict stock prices. # # Input graphs were created for spearman, Spearman, and Kendal Tau correlations\/coefficients from historical stock prices. Also, another &hellip; <\/p>\n<p><a class=\"more-link btn\" href=\"http:\/\/bangla.sitestree.com\/?p=76091\">Continue reading<\/a><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"categories":[182],"tags":[],"class_list":["post-76091","post","type-post","status-publish","format-standard","hentry","category---blog","item-wrap"],"jetpack_featured_media_url":"","jetpack_sharing_enabled":true,"jetpack-related-posts":[{"id":78242,"url":"http:\/\/bangla.sitestree.com\/?p=78242","url_meta":{"origin":76091,"position":0},"title":"Statistics for Data Analytics and Machine Learning Projects","author":"Sayed","date":"May 22, 2025","format":false,"excerpt":"\u2022Null Hypothesis \u2022[2] \u2022Paired t-test \u2022Unpaired t-test \u2022Pearson Correlation \u2022One Way: Analysis of variance \u2022Spearman Correlation \u2022Spearman \u2022Kendal Tau Coef \u2022Wilcoxon Sum test \u2022Basic EDA \u2022Mcnaimer\u2019s test \u2022Friedman test \u2022Kruskal-Wallis Test \u2022Two Way Analysis of variance \u2022K-Fold Cross Validation paired t-test \u2022Wilcoxon Signed Rank Test Data Analytics, Machine Learning, Data\u2026","rel":"","context":"In &quot;Analytics and Machine Learning Project Development&quot;","block_context":{"text":"Analytics and Machine Learning Project Development","link":"http:\/\/bangla.sitestree.com\/?cat=1974"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-37.png?resize=525%2C300 1.5x"},"classes":[]},{"id":76436,"url":"http:\/\/bangla.sitestree.com\/?p=76436","url_meta":{"origin":76091,"position":1},"title":"1. Libraries used for the project: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)","author":"Sayed","date":"December 4, 2024","format":false,"excerpt":"#import libraries import osimport pandas as pdimport math #Import Libraries for Graph, GNN, and GCN import stellargraph as sgfrom stellargraph import StellarGraphfrom stellargraph.layer import DeepGraphCNNfrom stellargraph.mapper import FullBatchNodeGeneratorfrom stellargraph.mapper import PaddedGraphGeneratorfrom stellargraph.layer import GCN #Machine Learnig related library Imports from tensorflow.keras import layers, optimizers, losses, metrics, Modelfrom sklearn import preprocessing,\u2026","rel":"","context":"In &quot;Code: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)&quot;","block_context":{"text":"Code: Predict Future Stock Price using Graph Theory, Machine Learning and Deep Learning)","link":"http:\/\/bangla.sitestree.com\/?cat=1969"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":16302,"url":"http:\/\/bangla.sitestree.com\/?p=16302","url_meta":{"origin":76091,"position":2},"title":"Graph Mining: Shared Nearest Neighbors : Clustering : Community Detection","author":"Sayed","date":"October 7, 2019","format":false,"excerpt":"Graph Mining: Shared Nearest Neighbors : Clustering : Community Detection Graph Mining: Shared Nearest Neighbors (SNN): Clustering : Community Detection: Learn by Finding Answers to the Following Questions. Will use SNN sometimes. What is one another name of the algorithm: Shared Nearest Neighbors? What is the purpose of the Algorithm:\u2026","rel":"","context":"In &quot;Graph Mining&quot;","block_context":{"text":"Graph Mining","link":"http:\/\/bangla.sitestree.com\/?cat=1905"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22885,"url":"http:\/\/bangla.sitestree.com\/?p=22885","url_meta":{"origin":76091,"position":3},"title":"Graph Mining: Shared Nearest Neighbors &#8211; Community Detection","author":"Sayed","date":"March 21, 2021","format":false,"excerpt":"Resources to Learn From Resources:Jarvis-Patrick Clusteringhttps:\/\/btluke.com\/jpclust.htmlEmpirical Comparison of Algorithms for Network Community Detectionhttps:\/\/cs.stanford.edu\/~jure\/pubs\/communities-www10.pdf Read the resources above to find answers. Shared Nearest Neighbors : Clustering : Community Detection Graph Mining: Shared Nearest Neighbors : Clustering : Community Detection Graph Mining: Shared Nearest Neighbors (SNN): Clustering : Community Detection: Learn by\u2026","rel":"","context":"In &quot;Graph Mining&quot;","block_context":{"text":"Graph Mining","link":"http:\/\/bangla.sitestree.com\/?cat=1905"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":22891,"url":"http:\/\/bangla.sitestree.com\/?p=22891","url_meta":{"origin":76091,"position":4},"title":"Graph Mining: Misc. Topics to Learn: Misc. Resources to Learn From","author":"Sayed","date":"March 21, 2021","format":false,"excerpt":"Graph Mining: Misc. Topics to Learn: Misc. Resources to Learn From Influence\/Virus\/Label Propagation Resources to learn fromPage A presentation on Influence\/Virus PropagationURL Big Data Graph Databases Resources to Learn FromPage Graph DatabaseURL Neo4jPage Big Data Graph Processing Resources to Learn FromPage ToolURL Techniques, Tools and Applications of Graph AnalyticURL Graph\u2026","rel":"","context":"In &quot;Graph Mining&quot;","block_context":{"text":"Graph Mining","link":"http:\/\/bangla.sitestree.com\/?cat=1905"},"img":{"alt_text":"","src":"","width":0,"height":0},"classes":[]},{"id":78178,"url":"http:\/\/bangla.sitestree.com\/?p=78178","url_meta":{"origin":76091,"position":5},"title":"How to Report (or Present) the outcome of your Analytics\/ML Project","author":"Sayed","date":"May 18, 2025","format":false,"excerpt":"Reporting and Analysis \u2022Examples \u2022Results section: Page 51: STOCK MARKET PREDICTION USING ENSEMBLE OF GRAPHTHEORY, MACHINE LEARNING AND DEEP LEARNING MODELS \u2022https:\/\/scholarworks.sjsu.edu\/cgi\/viewcontent.cgi?article=1692&context=etd_projects \u2022Check Results and Discussion sections \u2022https:\/\/arxiv.org\/ftp\/arxiv\/papers\/2203\/2203.06848.pdf \u2022A Comparative Study on Forecasting of Retail Sales May be complicated: Learning Context-Aware Classifier for Semantic Segmentation \u2022https:\/\/arxiv.org\/pdf\/2303.11633.pdf \u2022Learning Context-Aware Classifier for\u2026","rel":"","context":"In &quot;Analytics and Machine Learning Project Development&quot;","block_context":{"text":"Analytics and Machine Learning Project Development","link":"http:\/\/bangla.sitestree.com\/?cat=1974"},"img":{"alt_text":"","src":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-13.png?resize=350%2C200","width":350,"height":200,"srcset":"https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-13.png?resize=350%2C200 1x, https:\/\/i0.wp.com\/bangla.sitestree.com\/wp-content\/uploads\/2025\/05\/image-13.png?resize=525%2C300 1.5x"},"classes":[]}],"_links":{"self":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/76091","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=76091"}],"version-history":[{"count":0,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=\/wp\/v2\/posts\/76091\/revisions"}],"wp:attachment":[{"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=76091"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=76091"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/bangla.sitestree.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=76091"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}