顯示具有 自然語言 標籤的文章。 顯示所有文章
顯示具有 自然語言 標籤的文章。 顯示所有文章

2019年6月13日 星期四

[自然語言]用LSTM創作文字內容

步驟:
1. 字元等級神經語言模型(character-level neural language model):
使用LSTM層從文字庫中(尼采文章)以N個字元的字串作為輸入, 學習預測第N+1個字元的機率分佈,來建立字元等級神經語言模型(character-level neural language model).

2. 以逐一字元生成的方式產生文字資料
輸入N個測試字元的字串,用上面建立的字元等級神經語言模型預測第N+1個出現機率最高的字元,將此字元加入原先的測試字串末端,再送入語言模型......

import keras
import numpy as np

path = keras.utils.get_file(
    'nietzsche.txt',
    origin='https://s3.amazonaws.com/text-datasets/nietzsche.txt')
text = open(path).read().lower()
print('Corpus length:', len(text))



maxlen = 60
step = 3

sentences = []

next_chars = []

for i in range(0, len(text) - maxlen, step):
    sentences.append(text[i: i + maxlen]) #從i到i+maxlen-1
    next_chars.append(text[i + maxlen]) #i+maxlen

print('Number of sequences:', len(sentences))

#做出字元轉換成數字的字典
chars = sorted(list(set(text))) #列出所有出現在文章的字元
print('Unique characters:', len(chars))
char_indices = dict((char, chars.index(char)) for char in chars)

#將訓練的字元向量化
print('Vectorization...')
x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
    for t, char in enumerate(sentence):
        x[i, t, char_indices[char]] = 1
    y[i, char_indices[next_chars[i]]] = 1



from keras import layers

model = keras.models.Sequential()
model.add(layers.LSTM(128, input_shape=(maxlen, len(chars))))
model.add(layers.Dense(len(chars), activation='softmax')) #softmax for機率分佈


optimizer = keras.optimizers.RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)



#原始機率分佈受到溫度擾動而改變分佈機率,並傳回機率最大值的索引
def sample(preds, temperature=1.0):
    preds = np.asarray(preds).astype('float64')
    preds = np.log(preds) / temperature
    exp_preds = np.exp(preds)
    preds = exp_preds / np.sum(exp_preds)
    probas = np.random.multinomial(1, preds, 1) #丟一次骰子作為一次實驗,每一面出現機率為preds,回傳每個字元出現的“次數”串列
    return np.argmax(probas) #回傳結果陣列最大的索引值


import random
import sys

for epoch in range(1, 60):
    print('epoch', epoch)
    model.fit(x, y, batch_size=128, epochs=1)
    start_index = random.randint(0, len(text) - maxlen - 1)
    generated_text = text[start_index: start_index + maxlen]
    print('--- Generating with seed: "' + generated_text + '"')
    for temperature in [0.2, 0.5, 1.0, 1.2]:
        print('------ temperature:', temperature)
        sys.stdout.write(generated_text)  #印出generated_text
        

        #找出接下來最可能出現的400個字元
        for i in range(400):
            #將測試字串sampled向量化
            sampled = np.zeros((1, maxlen, len(chars)))
            for t, char in enumerate(generated_text):
                sampled[0, t, char_indices[char]] = 1.
            #將sampled丟入模型進預測,並不顯示過程(verbose=0)
            preds = model.predict(sampled, verbose=0)[0]
            #將預測的出現字元機率進行溫度擾動,回傳這個出現字元的索引值
            next_index = sample(preds, temperature)
            #將字元索引值帶入字典,取得字元
            next_char = chars[next_index]
            #將字元接到輸入文字的最下方,並刪除第一個字元
            generated_text += next_char
            generated_text = generated_text[1:]

            sys.stdout.write(next_char) #印出generated_text


#第一次訓練
epoch 1
Epoch 1/1
200278/200278 [==============================] - 472s 2ms/step - loss: 1.9867
--- Generating with seed: "ie et sans esprit!

#將“in these later ages, which may be”丟入訓練模型,用不同的溫度來創作文章

229. in these later ages, which may be "
------ temperature: 0.2
ie et sans esprit!
229. in these later ages, which may be the such the still and the sure and and still the present the sure the man and the presenter the for the still and the sure of the from the still the man be the string the man becount and still the the the the sure the still that the soul and a more of the sure the still the stright the stright and the still the sure the sure and the moral the still the sure the sure and the still the sure and the

------ temperature: 0.5
l the still the sure the sure and the still the sure and the perhaps so this disto the philosopher that the string and will the super; and still the same with the desilse of the shill ana
suld conter the free solition of the than the sure for the sure desilse of presentate of the soulh and the for the histances to litely than this incression, and from the preale of contention, in the precestion, that the present to the discienteness than the is and are thi

------ temperature: 1.0
hat the present to the discienteness than the is and are thing, the musf lattire tercies somolian of remord hister
dolecy, with men ye of suses, is and a
corstare--thas and sole of the consciencly to yees as lose. the denchsion in the fantific fan and stone, and trung with their cincolne, and spoled asted to som suef
agage well in regeraving of real the spirits
mistodicato high returdisming
powhing.--as yre? of sulf the doven weally froe it wat give wo le

2019年6月12日 星期三

[自然語言] 單字的one-hot encoding 把文字轉換為機器向量語言

做文件的向量化流程如下

  1. 將文件分解成小字元(token)
  2. 經由一個字典對照表將token編碼成數值向量
  3. 把這些數值向量打包成序列張量送入深度學習網路
其中token轉換為向量有兩種主要的方法:
  1. token的one-hot encoding
  2. 文字嵌入法(token embedding)


#one-hot encoding:


#1-1. token的one-hot encoding

import numpy as np
samples=['I want to be an machine learning expert.','I need to study a lot.']
token_index={}  #建立空字典儲存所有token和key
#建立token_index字典
for sample in samples:
    for word in sample.split():
        if word not in token_index:
            token_index[word]=len(token_index)+1
#將字串的token轉成向量
max_length=10
results=np.zeros(shape=(len(samples),max_length, max(token_index.values())+1))
for i, sample in enumerate(samples):
    for j, word in list(enumerate(sample.split()))[:max_length]:
        index=token_index.get(word)
        results[i,j,index]=1.


# 1-2. 字元的encoding
import string
samples=['I want to be an machine learning expert.','I need to study a lot.']
characters=string.printable #列出所有可以印出的字元
print(len(characters))
token_index=dict(zip(characters,range(1,len(characters)+1))) #把字串變成字典
max_length=50 #只翻譯樣本的前50個字元
results=np.zeros((len(samples),max_length,max(token_index.values())+1))
print(results.shape)
for i, sample in enumerate(samples):
    for j, character in enumerate(sample):
        index=token_index.get(character)
        results[i,j,index]=1.


# 1-3. 使用keras內建工具來做one-hot encoding
from keras.preprocessing.text import Tokenizer
samples=['I want to be an machine learning expert.','I need to study a lot.']
tokenizer=Tokenizer(num_words=1000)  #處理前1000個最常用單字
tokenizer.fit_on_texts(samples)
#將文件轉換成數字
sequences=tokenizer.texts_to_sequences(samples)
print(sequences)
#將文字轉換成向量
one_hot_results=tokenizer.texts_to_matrix(samples, mode='binary')#兩位元
print(one_hot_results)
#印出單字和索引對照表
word_index=tokenizer.word_index
print('found %s unique tokens.'%(word_index))

2019年6月9日 星期日

[自然語言] 文字嵌入(word embedding)法在機器情感判別的實例---IMDB影評分類

文字嵌入法(token embedding):

相較於one-hot encoding的二進位稀疏向量(大部分由0組成)且具有非常高維度特性; token embedding取得的是低維度的浮點數向量,適用於20000個以上的單字處理
有兩種方法建立文字嵌入向量:
  1. 用Embedding layer學習文字嵌入向量
  2. 用其他machine learning已經建立好的文字嵌入向量

實際案例:
下載IMDB的影評的[文字]與[評價(值得看(1)or不值得看(0))],並用Keras建立模型

方法:
  1. 將下載的檔案解壓縮,資料夾分成training, test; training和test資料夾內又有pos和neg資料夾分別儲存正面和負面的文字檔.
  2. 依序打開文字檔,並把它讀入變成陣列
  3. 把文字陣列用keras.preprocessing.text.Tokenizer轉換成數字陣列
  4. 把不足100個元素的陣列補成長度為100的陣列
  5. 建立Keras模型
  6. 視覺化訓練過程
  7. 測試模型的準確度
  8. 儲存模型
import urllib.request
import os
import tarfile
import numpy as np
url='http://ai.stanford.edu/~amaas/data/sentiment/aclImdb_v1.tar.gz'
filepath='data/aclImdb_v1.tar.gz'
#下載壓縮檔案
if not os.path.isfile(filepath):
    result=urllib.request.urlretrieve(url,filepath)
    print('download',result)
#解壓檔案
if not os.path.exists('data/aclImdb'):
    tfile=tarfile.open(filepath,'r:gz')
    result=tfile.extractall('data/')
from keras.preprocessing import sequence #匯入把陣列補齊的module
from keras.preprocessing.text import Tokenizer  #匯入建立字典的module
import re  #匯入處理文字的module
def rm_tags(text):
    re_tag=re.compile(r'<[^>]+>')
    return re_tag.sub('',text)
#整理資料夾的檔案文字成陣列輸出
import os
def read_files(filetype):
    path='data/aclImdb/'
    file_list=[]
    
    positive_path=path+filetype+'/pos/'
    for f in os.listdir(positive_path):
        file_list+=[positive_path+f]
        
    negative_path=path+filetype+'/neg/'
    for f in os.listdir(negative_path):
        file_list+=[negative_path+f]
    
    print('read',filetype,'files:',len(file_list))
    
    all_labels=[1]*12500+[0]*12500
    
    all_texts=[]
    for fi in file_list:
        with open(fi,encoding='utf8') as file_input:
            all_texts += [rm_tags(''.join(file_input.readlines()))]
            
    return all_labels,all_texts
y_train,train_text=read_files('train')
y_test,test_text=read_files('test')
#建立字典Token
token=Tokenizer(num_words=2000)
token.fit_on_texts(train_text)
print(token.word_index)
#把影評轉換成數字
x_train_seq=token.texts_to_sequences(train_text)
x_test_seq=token.texts_to_sequences(test_text)
#把每一筆數字list的長度都設成100,不足100的補0
x_train=sequence.pad_sequences(x_train_seq,maxlen=100)
x_test=sequence.pad_sequences(x_test_seq,maxlen=100)
#建立模型
#使用Embedding把數字list轉乘向量list
from keras.models import Sequential
from keras.layers import Dense, Dropout, Activation, Flatten,SimpleRNN
from keras.layers.embeddings import Embedding
model=Sequential()
model.add(Embedding(output_dim=32,input_dim=2000,input_length=100))
#output_dim: 輸出維度; input_dim: 字典數; input_length: 輸入每陣列長度 
model.add(Dropout(0.2))
#embedding嵌入向量是3D向量,需要用Flatten轉換為2D向量 (第0軸不變,第1軸為其他維度相乘)
model.add(Flatten())
model.add(Dense(units=256,activation='relu'))
model.add(Dropout(0.25))
model.add(Dense(units=1,activation='sigmoid'))
model.summary()
model.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy']) train_history=model.fit(x_train,y_train, validation_split=0.1, epochs=30,batch_size=30,verbose=2) #視覺化訓練過程 import matplotlib.pyplot as plt def show_train_history(train_history,train,validation): plt.plot(train_history.history[train]) plt.plot(train_history.history[validation]) plt.title('Train History') plt.ylabel(train) plt.xlabel('Epoch') plt.legend(['train','validation'],loc='upper left') plt.show() show_train_history(train_history,'acc','val_acc') #若訓練(train)的準確度一直增加而驗證(validation)的準確度沒有一直增加則可能是overfit
#儲存keras model
model.save('my_model.h5')
#刪除載入的keras model
del model  
from keras.models import load_model
model = load_model('my_model.h5')
model.summary()
#測試keras模型的準確度 y_test=np.transpose(y_test) scores=model.evaluate(x_test,y_test,verbose=1) #取得測試準確度 scores[1] #Add RNN model insize #建立模型 #使用Embedding把數字list轉乘向量list from keras.models import Sequential from keras.layers import Dense, Dropout, Activation, Flatten from keras.layers.embeddings import Embedding model=Sequential() model.add(Embedding(output_dim=32,input_dim=2000,input_length=100)) model.add(Dropout(0.2)) model.add(SimpleRNN(units=16)) #model.add(Flatten()) model.add(Dense(units=256,activation='relu')) model.add(Dropout(0.25)) model.add(Dense(units=1,activation='sigmoid')) model.compile(loss='binary_crossentropy', optimizer='adam',metrics=['accuracy']) train_history=model.fit(x_train,y_train, validation_split=0.1, epochs=30,batch_size=30,verbose=2) (train_history,'acc','val_acc') #若訓練(train)的準確度一直增加而驗證(validation)的準確度沒有一直增加則可能是overfit


#儲存keras model
model.save('my_model_RNN.h5')
scores=model.evaluate(x_test,y_test,verbose=1)
scores[1]

三倍槓桿和一倍槓桿的長期定期定額報酬率分析

  以下是中國,美國股票債卷的三倍槓桿和一倍槓桿ETF分析.可以發現,三倍槓桿在下跌時期的跌幅遠比一倍槓桿的多 .且從時間軸來看,三倍槓桿由於下跌力道較強,因此會把之前的漲幅都吃掉,所以對於長期上身的市場,例如美國科技股,由於上升時間遠比下跌時間長,所以持有TQQQ的長期回報率會...