Python: Read RBC Canada: Mastercard Statement Data into CSV file

Sure, you can use tools such as Adobe Acrobat to convert PDF statements to csv/xls files. This is a simple approach that you might be able to use when you do not have access to any such tools.

How can this help? If you were trying to calculate based on the transactions. One application can be calculating your business or personal expenses. Also, if you are writing a tax or financial software, similar can be useful. For sure, tools and programming languages are available to read PDF files directly than the approach I mentioned here.

Steps:

Download the statement.
Copy the text (only the transactions) into a text file
The use the code provided here or similar to read the text file and create a CSV file
You can write another Python code to merge multiple csv files
Using the combined CSV file, you can calculate your yearly categorized expenses or similar : for example

Sample Transaction Data
‘DEC 21 DEC 24 PAYMENT – THANK YOU / PAIEMENT – MERCI\n75105398358619984366202\n-$60.00\nJAN 10 JAN 10 PAYMENT – THANK YOU / PAIEMENT – MERCI\n75105399010619980541202\n-$21.82\nJAN 10 JAN 10 PURCHASE INTEREST 19.99% \n12312312312\n-$0.82\n\n\nDEC 10 DEC 12 PRESTO VIC PARK RELOAD EAST YORK ON\n55134428345800179063337\n$10.25\nDEC 13 DEC 14 TIM HORTONS 0670 QTH TORONTO ON\n55419218347200311042837\n$1.80\n

Note:
Assume that each transaction spans into three lines when you copy from a pdf file to a text file. The lines with interest charges might span into only one line. You can convert that into 3 lines before executing the code on it.

DEC 21 DEC 24 PAYMENT – THANK YOU / PAIEMENT – MERCI
75105398358619984366202
-$60.00
JAN 10 JAN 10 PAYMENT – THANK YOU / PAIEMENT – MERCI
75105399010619980541202
-$21.82
JAN 10 JAN 10 PURCHASE INTEREST 19.99%
12312312312
-$0.82

The Code:
#!/usr/bin/env python
# coding: utf-8

# In[5]:

file = open(‘dec2018-jan2019.txt’,’r’);

import os.path
import re

# open the file for reading
path = ‘dec2018-jan2019.txt’
if os.path.exists(path):
# print(‘debug: exists’)
file = open(‘dec2018-jan2019.txt’,’r’)
else:
print(‘Fatal error: file does not exist’)
exit(1)

# In[6]:

fileContent = file.read()
fileContent

# In[ ]:

#f = open("dec2018-jan2019.out.txt", "a");
#f.write("Now the file has more content!")
#f.close()

# In[3]:

one_line = ”;
count = 0;
f = open("dec2018-jan2019.out.txt.csv", "a");
file = open(‘dec2018-jan2019.txt’,’r’)
for line in file:
#print(line);
line = line.rstrip("\n");
if line !=”:
count = count + 1;

if (count % 3 == 1):
line = line[0:6] + "," + line[6:13] + "," + line[13:];

if (count % 3 == 2):
line = line + "," ;

if (count % 3 == 0):
#print(line[0]);
if line[0] == ‘-‘:
line = line[1:];
if (line[0] != ‘$’ ):
print(‘Warning possible mistake’)

#print(count);
one_line = one_line + line;

if (count % 3 == 0):
one_line = one_line + "\r\n";
print(one_line);
f.write(one_line);
one_line = ”;

#print(‘–‘);

f.close();

# In[ ]:

*** ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***
Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Training Courses: http://Training.SitesTree.com

Facebook Groups/Forums to discuss (Q & A):
https://www.facebook.com/banglasalearningschool
https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

Misc Stock Market: Crash: Bottom : Bear: Recession

Shiller PE Ratio by Month

https://www.multpl.com/shiller-pe/table/by-month

PE Ratio (TTM)

https://www.profitspi.com/stock/view.aspx?v=stock-chart&uv=101055

Moving Averages: S&P 500 Index

https://www.barchart.com/stocks/quotes/$SPX/technical-analysis

Hang/San Index

https://www.hsi.com.hk/eng

"we are likely to see a global recession in the coming quarters."

https://www.greaterfool.ca/2020/03/14/anatomy-of-bear-markets/comment-page-1/?unapproved=694043&moderation-hash=2a5261659bd9bc483fae73ca46c67007#comment-694043

"Therefore, we don’t believe any real bottom will setup before May, June or July of 2020 – after Q2 earnings are announced and contingent on the virus event subsiding and earnings starting to recover. Otherwise, we could be “searching for a bottom” for quite a while yet."

https://www.thetechnicaltraders.com/wheres-the-bottom-the-question-everyone-is-asking/

***. ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***
Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Training Courses: http://Training.SitesTree.com

Facebook Groups/Forums to discuss (Q & A):
https://www.facebook.com/banglasalearningschool
https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

Coronavirus and the Stock Market

"Finally, the news media will never keep things in perspective. Ignore them." https://www.greaterfool.ca/wp-content/uploads/2020/03/DOUG-2.jpg?x70502 https://www.greaterfool.ca/wp-content/uploads/2020/03/DOUG-3.jpg?x70502 https://www.greaterfool.ca/wp-content/uploads/2020/03/DOUG-4.jpg?x70502 https://www.greaterfool.ca/wp-content/uploads/2020/03/DOUG-5.png?x70502

Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***
Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Training Courses: http://Training.SitesTree.com

Facebook Groups/Forums to discuss (Q & A):
https://www.facebook.com/banglasalearningschool
https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

Project: Optimization

Portfolio Optimization with Linear and Fixed Costs

Report: https://github.com/chsasank/CVX/blob/master/Report/report.pdf

Project: https://github.com/chsasank/CVX

Trajectory Optimization

https://github.com/zi-w/6.832-project

CourseSchedulingOptimization

https://github.com/BenKang34/CourseSchedulingOptimization

Local search and optimization algorithms in AI

https://github.com/aliarjomandbigdeli/local-search-and-optimization

CS524-Group-Assignment-Optimization

https://github.com/xiaohk/CS524-Group-Assignment-Optimization

TOP – Timetable Optimisation Project

https://people.orie.cornell.edu/mru8/orie6326/project.html

Website Performance Optimization portfolio project

https://github.com/baker-natalie/optimization-project

Target Tracking Projects on Github

https://github.com/topics/target-tracking

Tracking Multiple People in a Multi-Camera Environment

https://www.epfl.ch/labs/cvlab/research/research-surv/research-body-surv-index-php

Appendix:

Optimization Course: https://github.com/MichielStock/SelectedTopicsOptimization

ORIE 7191: Topics in Optimization for Machine Learning

https://github.com/udellgroup/orie7191

Course

https://github.com/ssinhaleite/eth-mathematical-optimization-summary


*** ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

Python Libraries for Data Science esp. for NLP – Natural Language Processing

For NLP tasks, either you will come across these libraries or you will have to use many of these Python libraries.

import nltk
# tokenizer
nltk.download("punkt")

# stop words
nltk.download("stopwords")
from nltk.tokenize import TreebankWordTokenizer
from nltk.tokenize import WordPunctTokenizer
from nltk.tokenize import RegexpTokenizer
from nltk.tokenize import sent_tokenize
from nltk.corpus import stopwords
import os.path
import re

from nltk.tokenize import RegexpTokenizer
from nltk.corpus import stopwords
from nltk.tokenize import RegexpTokenizer
import nltk
from nltk.tokenize import TreebankWordTokenizer
from nltk.tokenize import WordPunctTokenizer
from nltk.tokenize import word_tokenize
from nltk.corpus import stopwords
import nltk
from nltk.util import ngrams
from collections import Counter
from nltk.collocations import *
from nltk.tokenize import word_tokenize
from nltk.probability import ConditionalFreqDist, FreqDist
from nltk.probability import ConditionalProbDist, LaplaceProbDist
from nltk.corpus import stopwords
from nltk.metrics import TrigramAssocMeasures
from nltk.tokenize import TreebankWordTokenizer
#from nltk.probability import *
import math
from nltk.metrics import TrigramAssocMeasures
from nltk.metrics import BigramAssocMeasures
from nltk.metrics import BigramAssocMeasures

import math
import random
from collections import Counter, defaultdict

import nltk
nltk.download("gutenberg")

from nltk.corpus import gutenberg
from nltk.util import ngrams

import csv
from numpy import array
from numpy import asarray
from numpy import zeros
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Embedding
import matplotlib.pyplot as plt

import nltk

nltk.download(‘averaged_perceptron_tagger’)

import nltk
from nltk.corpus import treebank

import nltk
from nltk.tag import StanfordNERTagger
from nltk.metrics.scores import accuracy

import nltk
from nltk.corpus import treebank
from nltk.classify import maxent

from __future__ import print_function, unicode_literals, division

import re
import itertools

from six.moves import map, zip

from nltk.probability import (
FreqDist,
ConditionalFreqDist,
ConditionalProbDist,
DictionaryProbDist,
DictionaryConditionalProbDist,
LidstoneProbDist,
MutableProbDist,
MLEProbDist,
RandomProbDist,
)
from nltk.metrics import accuracy
from nltk.util import LazyMap, unique_list
from nltk.compat import python_2_unicode_compatible
from nltk.tag.api import TaggerI

import matplotlib as mpl
import matplotlib.pyplot as plt

import numpy as np

from sklearn import datasets
from sklearn.mixture import GaussianMixture
from sklearn.model_selection import StratifiedKFold

import tensorflow as tf
import numpy as np
import random

import keras
from keras.layers import Dense
from keras import models
from keras import layers
from keras.layers import Activation, Dense

from keras import optimizers

from gensim.summarization import summarize

from gensim.summarization import keywords
from sklearn.datasets import fetch_20newsgroups
from nltk.corpus import wordnet
from nltk.stem import WordNetLemmatizer
import string
from sklearn.feature_extraction.text import CountVectorizer
from sklearn.decomposition import LatentDirichletAllocation
from sklearn.decomposition import TruncatedSVD
nltk.download("averaged_perceptron_tagger")
nltk.download(‘tagsets’)
nltk.help.upenn_tagset("JJS")
nltk.download(‘treebank’)
nltk.download(‘ brown’)
nltk.download(‘universal_tagset’)
import nltk
from nltk.corpus import treebank
import nltk
from nltk.corpus import treebank
from bs4 import BeautifulSoup # For HTML parsing
import urllib # Website connections
import re # Regular expressions
from time import sleep # To prevent overwhelming the server between connections
from collections import Counter # Keep track of our term counts
from nltk.corpus import stopwords # Filter out stopwords, such as ‘the’, ‘or’, ‘and’
import pandas as pd # For converting results to a dataframe and bar chart plots
import numpy as np
import copy
%matplotlib inline
from sklearn.mixture import GaussianMixture
from sklearn.feature_extraction.text import TfidfVectorizer
import operator

from sklearn.datasets import load_files
import nltk
import string
from sklearn.feature_extraction.text import CountVectorizer
from nltk.stem import PorterStemmer
from sklearn.naive_bayes import MultinomialNB
from sklearn.metrics import precision_recall_fscore_support
import pandas as pd
import nltk
from nltk.corpus import treebank
import tensorflow as tf
import keras
from keras.layers import Dense
from keras import models
from keras import Sequential

from numpy import array
from keras.preprocessing.text import one_hot
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.embeddings import Embedding
from sklearn.datasets import load_files
import nltk
import string
from sklearn.feature_extraction.text import CountVectorizer
from nltk.stem import PorterStemmer
from keras import models
from numpy import array
from keras.preprocessing.text import one_hot

from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers.embeddings import Embedding
from keras.preprocessing.text import one_hot

from numpy import array
from numpy import asarray
from numpy import zeros
import pandas as pd
from keras.preprocessing.text import Tokenizer
from keras.preprocessing.sequence import pad_sequences
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import Flatten
from keras.layers import Embedding

from keras.utils import to_categorical
from sklearn.model_selection import train_test_split

from keras.models import Model, Input
from keras.layers import LSTM, Embedding, Dense, TimeDistributed, Dropout, Bidirectional
from keras.layers.merge import add

from seqeval.metrics import precision_score, recall_score, f1_score, classification_report

from keras.models import Model, Input
from keras.layers import LSTM, Embedding, Dense, TimeDistributed, Dropout, Bidirectional
from keras.layers.merge import add
from keras import models, layers

from numpy import zeros

*** ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Our free or paid training events: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to occasional free and/or low cost online/offline training or charitable/non-profit work in the education/health/social service sector, you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).

Misc. Plot – 4

Ref: https://www.wolframalpha.com/

Misc Plots – 3

Misc Plots

Misc Plots:
Ref: https://www.wolframalpha.com/


***. ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Our free or paid training events: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to occasional free and/or low cost online/offline training or charitable/non-profit work in the education/health/social service sector, you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).

Misc. Plots

Ref: https://www.wolframalpha.com/

—-
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Our free or paid training events: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to occasional free and/or low cost online/offline training or charitable/non-profit work in the education/health/social service sector, you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).

Euclidean Norm of a Matrix

Euclidean Norm of a Matrix

Ref: http://mathworld.wolfram.com/FrobeniusNorm.html

***. ***. ***
Note: Older short-notes from this site are posted on Medium: https://medium.com/@SayedAhmedCanada

*** . *** *** . *** . *** . ***

Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
Online and Offline Training: http://Training.SitesTree.com (Also, can be free and low cost sometimes)

Facebook Group/Form to discuss (Q & A): https://www.facebook.com/banglasalearningschool

Our free or paid training events: https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

If you want to contribute to occasional free and/or low cost online/offline training or charitable/non-profit work in the education/health/social service sector, you can financially contribute to: safoundation at salearningschool.com using Paypal or Credit Card (on http://sitestree.com/training/enrol/index.php?id=114 ).