deep learning methods and text mining related concepts for Alibaba interview 2

word2vec & GloVe

递归神经网络(Re- cursive Neural Network)的相关工作。该方法被证实在构建句子级语义时较为有 效。然而,递归神经网络需要按照一个树形结构来构建句子的语义,其性能依 赖于构建文本树的精度。而且,构建这棵树需要至少 O(n2) 的时间复杂度,其 中 n 表示句子的长度。当模型在处理长句子或者文档时,所花费的时间往往是 不可接受的。更进一步地,在做文档表示时,两个句子之间的关系不一定能构 成树形结构。因此递归神经网络可能不适合构建长句子或者文档的语义。 

循环神经网络(Recurrent Neural Network)可以在 O(n) 时间内构建文本的 语义 [26]。该模型逐词处理整个文档,并把所有上文的语义保存到一个固定大 小的隐藏层中。循环神经网络的优势在于它可以更好地捕捉上下文信息,对长 距离的上下文信息进行建模。然而,循环神经网络是一个有偏的模型,如对于 正向的循环神经网络而言,文本中靠后的词相对靠前的词占据了更主导的地位。 由于这一语义偏置的特性,循环神经网络在构建整个文本的语义时,会更多地 包含文本后面部分的信息。但是实际上并非所有文本的重点都放在最后,这可 能会影响其生成的语义表示的精确度。
为了解决语义偏置的问题,有人提出用卷积神经网络(Convolutional Neural Network)来构建文本语义 [18]。卷积神经网络利用最大池化技术能从文本中找 出最有用的文本片段,其复杂度也是 O(n)。因此卷积神经网络在构建文本语义 时有更大的潜力。然而,现有卷积神经网络的模型总是使用比较简单的卷积核, 如固定窗口 [18, 46]。在使用这类模型时,如何确定窗口大小是一个关键问题。 当窗口太小时,可能导致上下文信息保留不足,难以对词进行精确刻画;而当 窗口太大时,会导致参数过多,增加模型优化难度。因此,需要考虑,如何构 建模型,才能更好地捕获上下文信息,减少选择窗口大小带来的困难。并以此 为基础来更好地完成文本分类的任务。 

deep learning methods and text mining related concepts for Alibaba interview

CNN(卷积神经网络)、RNN(循环神经网络)、DNN(深度神经网络)的内部网络结构区别

https://www.zhihu.com/question/34681168

http://www.cnblogs.com/pinard/p/6418668.html
http://blog.csdn.net/Real_Myth/article/details/53780314
https://www.jiqizhixin.com/articles/2017-03-30-4
http://www.jianshu.com/p/9dc9f41f0b29
https://www.jiqizhixin.com/articles/2017-11-04-3

反向传播：
https://zhuanlan.zhihu.com/p/24801814
http://blog.csdn.net/pennyliang/article/details/6695355
https://www.zhihu.com/question/27239198/answer/154510111
https://zhuanlan.zhihu.com/p/25081671

max pooling:
https://www.zhihu.com/question/23437871
http://blog.csdn.net/jiejinquanil/article/details/50042791
http://blog.csdn.net/pangjiuzala/article/details/71840947

LSTM:
https://www.zhihu.com/question/41949741
http://blog.csdn.net/jerr__y/article/details/58598296

machine translation:
https://zhuanlan.zhihu.com/p/25366912

install tensorflow in Saturn (ILS server)

Saturn 的server是Red Hat 的，所以参考https://jiafulow.github.io/blog/2018/07/10/install-tensorflow-with-gpu-on-redhat/ 来安装Tensorflow-GPU.

For Ubuntu server,
为了run python 3.6 + tensorflow GPU version in ILS saturn server, 具体步骤：

一 准备工作：
1. 安装anaconda 3。在.bashrc 里面加入
PATH=/home/gao27/anaconda/bin:$PATH
2. 确定你的Python version 3.6

二 安装cuda 和 cudnn
1. 因为我们没有权限在cuda里面直接安装，所以我们首先通过which nvcc 找到cuda 包的所在地，然后把cuda整个包都拷贝到一个我们有权限操作的地方。 cuda版本需要8.0
2. 下载cudnn 5.1 版本，解压后会有两个文件夹lib64 和 include。 把里面的文件对应加到cuda对应的lib64 和 include里面 （不同版本的tensorflow 对应不同的cudnn. tensorflow 1.4 对应6.0. ）
3. 在.bashrc 里面加入如下路径
#cuda settings
export PATH=$HOME/cuda/bin:$PATH
export CPATH=$HOME/cuda/include:$CPATH
export LIBRARY_PATH=$HOME/cuda/lib64:$LIBRARY_PATH
export LD_LIBRARY_PATH=/$HOME/cuda/lib64:$LD_LIBRARY_PATH


最后要source ~/.bashrc
三 安装tensorflow GPU 版本通过Anaconda
如图：

网址如下：https://www.tensorflow.org/versions/r1.2/install/install_linux#python_36

然后activate tensorflow之后检验是否安装成功：

# Python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

https://blog.csdn.net/u014797226/article/details/80229887
https://blog.ailemon.me/2017/06/06/install-tensorflow-gpu-on-ubuntu-linux/
https://blog.csdn.net/u014797226/article/details/80229887

if you have trouble to install igraph in python

if you have trouble to install igraph in python, please check this page and install it: https://anaconda.org/marufr/python-igraph

some concepts may be useful for paper

笛卡尔乘积定义 笛卡尔乘积是指在数学中，两个集合X和Y的笛卡尓积（Cartesian product），又称直积，表示为X×Y，第一个对象是X的成员而第二个对象是Y的所有可能有序对的其中一个成员 。

HMM , factor graph 和 MRF是一个有向图一个无向图。
sum product algorithm 是belief propogration.  看Bishop的第八章

Some linux commend

log in time for all users:

lastlog

log in time for me:

lastlog | grep gao 

检查CPU使用率 从大到小排序

ps auxw --sort=%cpu

list all files:

ls -l

看file system 的大小和使用情况

df -h 

http://www.cnblogs.com/peida/archive/2012/10/30/2746968.html
http://mark-ztw.iteye.com/blog/1544367

查看文件大小，文件夹大小：du -sm *,du -h, du *, du -sm, du -h *,
查看文件多少行：wc -l 2018092121.tsv

[root@linux ~]# tar -cvf /tmp/etc.tar /etc  <==仅打包，不压缩！

[root@linux ~]# tar -czvf /tmp/etc.tar.gz /etc  <==打包后，以 gzip 压缩

[root@linux ~]# tar -cjvf /tmp/etc.tar.bz2 /etc  <==打包后，以 bzip2 压缩

把tar包分成多个小文件：split -b 40M home1.tar.bz2 "home.tar.bz2.part"
把多个小文件合并：cat home.tar.bz2.parta* > home2.tar.bz2

linux 并行执行。用& 和 wait 综合操作。
& 表示并行
wait可以再等前面执行完，串行回来
https://blog.csdn.net/gua___gua/article/details/48682665

如果remove 一个folder ,然后不出现一个一个文件删除的确认的话，需要用
rm -rf folder/
一个一个文件删除的话 用：
rm -r folder/

Shell cheat sheet:
https://devhints.io/bash

对于shell 来说，如果想下一个命令要在上一个命令完成之后才开始的话，需要在每行命令后面加上 && 符号 https://blog.csdn.net/ysdaniel/article/details/6127860

https://blog.csdn.net/tiwoo/article/details/51094913


有zsh 也有bash. 对应的配置文件为zshrc 和 bashrc. 为了查看到底机器用的是哪个shell, 用
echo $SHELL 来查看。

bashrc 和 bash_profile 的区别： https://justcoding.iteye.com/blog/2120331

Some notes for check python location, module, virtualenv

If you can't see the virtualenvwrapper.sh. First uninstall virtualenvwrapper. It will show messages about the location of the virtualenvwrapper.sh file.


Check all python' location

type -a python

首选要确定哪个版本的python 要使用（不同的python版本可能存在在不同的地方）
可以用module load python/2.7 用module查看有几个版本的python

首先install virtualenv:
pip install virtualenv

新建一个sandbox of virtual environment:

virtualenv --python=/l/python2.7/bin/python env1 (silo 上面 module load的 python 不是系统自带的，也不是anaconda上面的)

有些python 用的是anaconda的，所以需要指定python路径

把当前路径设为指定的virtual envir：

cd env1/

 source bin/activate

which python

pip list //看一下安装的包都有什么

pip freeze// python 所有依赖的包

退出虚拟环境：

deactivate

如果你的虚拟环境想用系统已有的包，when you create it, you need to use

virtualenv --system-site-packages venv




Otherwise, if you don't want to use system package, create it using:

virtualenv venv –no-site-packages (和以上的基本一样)



2.3. 指定python版本

可以使用-p PYTHON_EXE选项在创建虚拟环境的时候指定python版本

#创建python2.7虚拟环境
➜  Test git:(master) ✗ virtualenv -p /usr/bin/python2.7 ENV2.7
Running virtualenv with interpreter /usr/bin/python2.7
New python executable in ENV2.7/bin/python
Installing setuptools, pip...done.

#创建python3.4虚拟环境
➜  Test git:(master) ✗ virtualenv -p /usr/local/bin/python3.4 ENV3.4
Running virtualenv with interpreter /usr/local/bin/python3.4
Using base prefix '/Library/Frameworks/Python.framework/Versions/3.4'
New python executable in ENV3.4/bin/python3.4
Also creating executable in ENV3.4/bin/python
Installing setuptools, pip...done.

作者：Andrew_liu
链接：http://www.jianshu.com/p/08c657bd34f1
來源：简书
著作权归作者所有。商业转载请联系作者获得授权，非商业转载请注明出处。

使用virtualenvwrapper （更简单的方法）

http://codingpy.com/article/virtualenv-must-have-tool-for-python-development/

http://qicheng0211.blog.51cto.com/3958621/1561685

http://www.jianshu.com/p/44ab75fbaef2

http://liuzhijun.iteye.com/blog/1872241

man 在Linux中可以有文档查看的功能 i.e.
man ls
man python

Gephi

Useful links:
http://blog.csdn.net/golden1314521/article/details/44982023

乐理

全全半 全全全半 这个是一个音阶
大调开心 小调悲伤
小调是大调3，6，7的降半音

在大调音阶中，每个音与第一个音的距离都叫做完全或者大音程
1458是完全 2367是大

减《--完全《--增
减《--小《--大《--增

五度循环圈

fancy words for academic papers

For vocabulary list,  please see the link below:
https://docs.google.com/spreadsheets/d/19aresbKpFTNOIe_6QjDZBN7iYNAMLCeqypFdRLmCUjc/edit?usp=sharing

IR related methods

LSI, PLSI, TransE,TransR Trans 系列：http://www.sohu.com/a/116866488_465975

linear algebra lecture 21 09/07/2017

特征值和特征向量

Ax = lamda*x
what are x and lamda for prpojection matrix
any x in the plane, Px = x ; lamda = 1
any x perpendicular to the plane, Px = 0x, lamda = 0

How to solve Ax = lamda * x

(A-lamda*I)*x = 0
A-lamda*I 一定要singular,不然没解
所以行列式为0

如果矩阵+3I, eigenvector don't change, eigen value +3

linear algebra lecture 20 08/28/2017

A-1 = 1/det(A)*Ct
C 是A的 余子式矩阵

克莱默法则
Ax=b
x=A-1b = > 1/det(A)*Ct*b
计算量超级大

行列式的值相当于某个物体的体积
detA = volume of a box
先从一个单位立方体开始 => det(cube) = 1
然后不同方向stretch. 或者变形之类的。可以看看其他的tutorial.

linear algebra lecture 18 & 19 08/27/2017

lecture 18
行列式 det A |A|
性质：
 1.
det I = 1
det singular matrix  = 0

2.
exchange rows reverse the sign of det.
det Permutation matrix = -1/ 1 根据单位矩阵来的

|a b
c d | = ad-bc

3.
|ta tb
c d| = t |a b; c d|

4. if two rows are equal, the det is 0.

5. subtract l*row i from row k. det doesn't change (消元的感觉)
6. row of 0s => det A = 0
7.三角矩阵的det 是对角线的乘积
8. det A = 0 if when A is singular
9.
det AB = detA*det B
det A-1 = 1/det A

det A2 = (det A)2
det 2A= 2^n*det A
(like a volume)

10. |At| = |A|
|Ut*Lt| = |LU|
=>|Ut|*|Lt| = |L|*|U|


lecture 19


det A = sum (A1aA2b...Anw)
如果 a,b...w可以通过偶数次变换成为1，2，。。。n 那就是正数，

余子式：行列式中指定一个数，去掉所在行和所在列的剩余矩阵是其中一个余子式
cofactor

linear algebra 08/23/2017 lecture 17

标准正交Q为什么好

QTQ = [q1t;q2t...]*[q1,12...]= [1,0...] 单元矩阵

projection matrix
P = Q(QtQ)-1Qt = QQt (=1 if Q is square)

gram-schmidt 正交化

conjugate prior

very nice resource:
http://www.cnblogs.com/lifegoesonitself/p/3423588.html

linear algebra lecture 14 & 15 08/21/2017

lecture 14:
正交：
row space and null space 正交 orthogonal 因为A*x = 0
column space and null space of At 正交
Xt*y=0

subspace S is orthogonal to subspace T means that every vector in S is orthogonal to every vector in T.

Ax = b 可能没有solution
=> 变成这个更好 AtAx = Atb (best solution in this situation)

AtA不一定永远可逆

N(AtA) = N(A)
rank (AtA) = R(A)

AtA is invertible exactly if A has independent columns.


lecture 15：projections and least square
求b在a上面的投影p: (a,b是个向量)
正交 aT(b-xa) = 0
x = aTb/aTa
projection p = ax

p = a*aT/(aT*a) 是projection matrix
性质1：rank是1 因为是一行*一列得到的
性质2：Pt = P
性质3： P2 = P

Why project?
Because Ax = b may not have solution
solve Ax = p instead. p is the projection of b onto the column space. (等式右边是column space 的话肯定会有解)

AT(b-Ax_hat) = 0
=>ATAx = ATb

x = (ATA)-1ATb
p = A(ATA)-1ATb

projection matrix P =  A(ATA)-1AT (不能再简化了，因为A不一定可逆，可能吧不是square matrix)

least square fitting by a line:

linear algebra 08/18/2017 lecture 12

lecture 12:
graph:
node - edge graph A (column 是 node; row 是edge)
现在看A的 null space

Euler's formula:
#nodes-#edges+#loops = 1

lecture 13:

u,v,w in Rt 构成一个 5*3 u  rank = 3
N(u) = n-r = 0 只有零向量


r = m or r = n (full rank matrix )

linear algebra 08/17/2017 lecture 11

symmetric & upper triangular matrix = diagonal matrix.  dimension = 3  in 3*3 matrix

Sum: symmetric or upper triangular matrix : take anything in S and anything in U. 不是并集，是相加。 所以可以得到all matrix. dimension (S+U) = 9 in 3*3 matrix

dim(S) +dim(U) = dim(S&U) + dim(S+U)

rank 1 matrix
example : A is 2*3 matrix

rank 1 matrix 像积木，eg. 一个rank 4 的matrix可以分解成4个rank 1 matrix的相乘

subset of rank 1 matrix can't form a sub space (两个rank 1 matrix相加可能得到一个rank 2 matrix)

In R4, V = [v1,v2,v3,v4], S is all v in R4 to make v1+v2+v3+v4 = 0
=>Av = 0 A = [1,1,1,1]
S 是A的null space. RA = 1 => S = R(N(A)) = n-r = 4-1 = 3
三个特殊解找出来即可

graph {nodes,edges}

linear algebra letcure 9 & 10 08/16/2017

lecture 9:
independence:
vector x1,x2,....xn are independent, if no combination gives 0 vector (except the 0 combinations.)

c1x1+c2x2+...+cnxn is not 0, they are independent

vectors v1,...vl span a space means: the space consists of all combinations of those vectors.

Basis for a space is a set of vectors with 2 properties:

1. they are independent.
2. they span the space

Example :
space is R3
one basis: [1;0;0] [0;1;0] [0;0;1]

n vectors give basis if the n*n matrix is invertible.
(n个vector的一种组合可以得到单元矩阵，也就是单元基basis)

every basis for the space has the same number of vectors.
It is called the dimension of the space.

rank(A) = #pivot column = dimension of the column space

dimension of null space is the number of free variables. = n - r

lecture 10:

4 subspace of A(m*n)
C(A) column space ;Rm
N(A) null space ;Rn
R(A) row space = all combination of columns of At C(At) ;Rn
N(At) :null space of At. The left null space of A ;Rm


basis? dimension?

C(A)
dimension of C(A) = r (rank of the matrix)
basis: is the pivot columns of A

R(A)

dimension of R(A) = r

N(A)
basis: special solutions
dimension: n-r

N(At)
basis: special solutions
dimension: m - r
why call it left null space?
At*y = 0 => yt*A = 0t =>左零矩阵

[A,I]=>[R,E]

如何计算y?
E*A = R
从E的最后一行找，因为E*A最后会得到一个最后一行为0的矩阵

linear algebra 08/14/2017 lecture 7 & 8

lecture 7:
the rank of the matrix. 矩阵的秩

do elimination, find the pivot columns and free columns.
pivot variables + free variables = n (column number)

reduced row echelon form 是一种矩阵的继续简化

求解null space
解法1：produce null space matrix R*N=0
R是已经被简化的矩阵，变成[I,F;0;0] I 是单元矩阵
N=[-F;I]

解法2：求解null space.把free variable 其中一个设为1，其余free variable 为0. 然后求出一组解，

lecture 8:
Ax = b
solvable when b is in C(A)
If a combination of rows of A gives 0 rows,

to find a compete solutions

1.x particular (找一个特殊解)： set all free variables as 0
2. 再把其中一个free variable 换成1，其余的为0，求几组特殊解


The case of full column rank 
r = n

what that imply about the solutions? What is that mean?
No free variables
=> N(A) = {0} null space only have 零向量
solution to Ax = b
x = unique solution, if exists (0 or 1 solution)

full row rank means 

r = m
can solve Ax = b for every b.
free variables n -r = n-m 个
肯定可以得出一个最简矩阵R=[I F]

special case: r = m = n

R = I
=> N(A) = {0} null space only have 零向量

Ax = b

r= m < n 

R= [I;0] => 0  or 1 solution

r = m<n

R = [I F] => infinite solution

r<m, r<n
R = [I,F;0,0]
=> 0 or infinite solutions

linear algebra notes 08//13/2017 lecture 5 & 6

lecture 5:

LU 分解的意义：
http://blog.csdn.net/carrierlxksuper/article/details/8487276

permutation:
P-1 = Pt
一共n!个permutation matrix

transpose 的概念

symmetric matrix：
At = A

Rt*R is always symmetric.
WHY? Take transpose!
(RtR)t = Rt*Rtt = Rt*R  （和逆矩阵的公式很像 (AB)-1 = B-1A-1）
所以这个是symmetric的


Vector space
Example: R2 = all 2 dimensional vectors

a vector space inside R2 is : a subspace of R2
一定得过零点

subspace of R2
1. all of R2 (最大子空间)
2. any lines through [0;0]
3. zero vector only （最小子空间）
子空间 补充：https://www.zhihu.com/question/48849797

column space: columns in Rn. Add their combinations form a subspace . (满足定义)


lecture 6:

Ax=b 有解 当b in A的column spaces (根据定义来的)

主列

null space (零空间) of A
all vectors x to let Ax = 0
at least contains [0;0;0]

solutions to Ax = 0 always gives a subspace.

subspace 一定会过零点

MIT linear algebra course note

08/11/2017 lecture 1 & 2
1. permutation matrix
2. inverse matrix E*E-1 = I
3. identity matrix
4. 如何做矩阵变换。E*A=B 使得 第二行substract 第一行

08/12/2017 lecture 3&4
4种方法计算矩阵乘法：
1. 正常法，每个值都是1 row * 1 column
2. column way. A乘以B的每个column
3. row way. A的每个row 乘以B
4. A的每个column 乘以 B的每个row。加在一起
5. cut matrix into blocks。 然后把每个block之间进行运算

A的左逆 矩阵和右逆矩阵是相同的 if matrix is invertible (non singular)

Singular case (no inverse)
比如[1,3;2,6]

singular matrix can get 0 by some matrix x
Ax = 0 if A is singular matrix. x 是存在的

A times B的一列j，得到的是结果的C的一列j
A的一行j times B,得到的是结果的C的一行j

Gauss - Jordan (solve 2 equations at once)
[A,I]--->[I,A-1]




AB 的逆矩阵 = B-1A-1

矩阵的转置
(A-1)t = (At)-1
A 的逆矩阵的转置是A的转置的逆矩阵
A= LU
if there is no row exchange, the multiplers will directly go into L

how many operations on n *n matrix?
n^3

permutation:
可以在单元矩阵中变换

原矩阵乘以这个矩阵(permuatation matrix)表示第二行第三行互换

permutation matrix ： p-1 = pt 如果第二行和第三行 换了，那么再换一次第二行和第三行就是还是原来的矩阵

置换矩阵（Permutation matrix）：矩阵的每一行和每一列的元素中只有一个1，其余元素都为0。（不严谨的解释）

转置矩阵（Transpose matrix）：矩阵的行变成对应的列，矩阵的列变成对应的行。（不严谨的直白解释）

性质：置换矩阵(P)的逆是其（置换矩阵自己的）转置(T)，

即：P^(-1)= P^T

如：3×3的置换矩阵群（共3! = 6个，补充4×4的置换矩阵共4! = 4×3×2×1 = 24个）

1 0 0 | 0 1 0

0 1 0 | 1 0 0

0 0 1 | 0 0 1

----------------

0 0 1 | 1 0 0  

0 1 0 | 0 0 1 

1 0 0 | 0 1 0 

置换矩阵的逆=置换矩阵的转置（上面4个置换矩阵的转置矩阵都是自身，又因为他们的逆=他们的转置，所以他们的逆=自身）

0 1 0 | 0 0 1 

0 0 1 | 1 0 0 

1 0 0 | 0 1 0 

置换矩阵的逆=置换矩阵的转置（这两个矩阵的转置=对方，因此这两个矩阵的逆=对方）

LU分解

LU分解是指，把矩阵分解为 消元矩阵E 的逆操作 L（下三角）

和 A 被消元后的结果矩阵 U（上三角）的乘积.

• LU分解不借助 行交换的关键是是 中间不能出现 0 pivot
• 但是对最后一个 pivot 没有 非0的要求
• 因为奇异矩阵也有LU分解
• 
  

http://blog.csdn.net/qq_25834201/article/details/59748093
http://blog.csdn.net/lc013/article/details/52244464
https://github.com/mebusy/notes/blob/master/dev_notes/LinearAlgebra.md

quickly log in remote sever

reference:
http://blog.csdn.net/leexide/article/details/17252369
https://my.oschina.net/aiguozhe/blog/33994

process:
local server:
1. ssh-keygen -t rsa (连续三次回车,即在本地生成了公钥和私钥,不设置密码)
2. ssh root@172.24.253.2 "mkdir .ssh;chmod 0700 .ssh" (需要输入密码， 注:必须将.ssh的权限设为700) (if remote server doesn't have ssh file)
3. scp ~/.ssh/id_rsa.pub root@172.24.253.2:.ssh/id_rsa.pub (需要输入密码)

remote server:
1. 
cat id_rsa.pub >> ~/.ssh/authorized_keys

2. 

 chmod 600 authorized_keys
3.
 chmod 700 -R ~/.ssh

注意都要在对应.ssh 文件目录下进行

python write to file buffer

https://stackoverflow.com/questions/18984092/python-2-7-write-to-file-instantly
https://stackoverflow.com/questions/3167494/how-often-does-python-flush-to-a-file
Use flush() or

For file operations, Python uses the operating system's default buffering unless you configure it do otherwise. You can specify a buffer size, unbuffered, or line buffered.

For example, the open function takes a buffer size argument.

http://docs.python.org/library/functions.html#open

"The optional buffering argument specifies the file’s desired buffer size: 0 means unbuffered, 1 means line buffered, any other positive value means use a buffer of (approximately) that size. A negative buffering means to use the system default, which is usually line buffered for tty devices and fully buffered for other files. If omitted, the system default is used."

bufsize = 0
f = open('file.txt', 'w', bufsize)

shareimprove this answer

Pages in Canvas

add code in Canvas
<p><code>java -version </code></p>
<pre><code>java -version </code></pre>

add github gist to html:
<iframe title="Code Embed" src="https://gist.github.com/RoyZhengGao/16c8a02c7d1fc90d616cc8f731b31d34.pibb" width="675" height="675" allowfullscreen="allowfullscreen" webkitallowfullscreen="webkitallowfullscreen" mozallowfullscreen="mozallowfullscreen"></iframe>

Information Theory, Inference and Learning Algorithms Reading

1. preface

 binary symmetric channe

l

 Information theory and coding theory oer an alternative (and much more exciting)

approach: we accept the given noisy channel as it is and add communication

systems  to it so that we can detect and correct the errors introduced by

the channel. As shown in gure 1.6, we add an encoder  before the channel and

a decoder  after it.

Majority-vote

decoding algorithm

chapter 4:The Source Coding Theorem

html format

add code to html
<code>int</code>

use iframe + github gist to embed long code in Canvas

<p class="p1"><span class="s1"> <iframe title="Code Embed" src="https://gist.github.com/RoyZhengGao/16c8a02c7d1fc90d616cc8f731b31d34.pibb" width="675" height="675" allowfullscreen="allowfullscreen" webkitallowfullscreen="webkitallowfullscreen" mozallowfullscreen="mozallowfullscreen"></iframe></span></p>

tensorflow

activate tensorflow:

source ~/Documents/tensorflow/bin/activate


deactivate tensorflow:

deactivate

Use anaconda in the server

every time to use anaconda pip, you need to execute the command first to find the path:




export PATH=/home/gao27/anaconda/bin:$PATH
或者把这行代码写入~/.bashrc 中，然后 source ~/.bashrc


install Anaconda in server
https://docs.anaconda.com/anaconda/install/linux


install pip directly:
https://pip.pypa.io/en/stable/installing/#id7

play with IU server

which virtualenv

virtualenv venv

module load python/2.7

pip install gensim

  156  virtualenv venv

  157  pip install gensim virtualenv venv

  158  virtualenv venv

  159  virtualenv

  160  pwd

  161  pip install virtualenv

  162  free -g

  163  history

  164  virtualenv


man module 

YY suggestions about how to do research

1. use git hub, push figures in it, 4 figures a paper. play with toy sample
2. do toy sample,  use Tex
for edge2vec thing:
1. debiashing hubs
link random walk to nodes with less degree
2. when links goes to the hub, it tend to random walk to the other links close to it.

python 2.7 and 3.5

http://conda.pydata.org/docs/py2or3.html

$ conda create -n py35 python=3.5 anaconda

#

# To activate this environment, use:

# $ source activate py35 (source activate py3)

#

# To deactivate this environment, use:

# $ source deactivate

#