Text Generation - LSTM
• 47 min read
Credit: Code from https://github.com/jeffheaton/t81_558_deep_learning
try:
%tensorflow_version 2.x
COLAB = True
print("Note: using Google CoLab")
except:
print("Note: not using Google CoLab")
COLAB = False
Note: using Google CoLab
from tensorflow.keras.callbacks import LambdaCallback
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import LSTM
from tensorflow.keras.optimizers import RMSprop
from tensorflow.keras.utils import get_file
import numpy as np
import random
import sys
import io
import requests
import re
r = requests.get("https://www.gutenberg.org/cache/epub/600/pg600.txt")
raw_text = r.text
print(raw_text[0:1000])
Project Gutenberg's Notes from the Underground, by Feodor Dostoevsky
This eBook is for the use of anyone anywhere at no cost and with
almost no restrictions whatsoever. You may copy it, give it away or
re-use it under the terms of the Project Gutenberg License included
with this eBook or online at www.gutenberg.net
Title: Notes from the Underground
Author: Feodor Dostoevsky
Posting Date: September 13, 2008 [EBook #600]
Release Date: July, 1996
Language: English
*** START OF THIS PROJECT GUTENBERG EBOOK NOTES FROM THE UNDERGROUND ***
Produced by Judith Boss. HTML version by Al Haines.
Notes from the Underground
FYODOR DOSTOYEVSKY
PART I
Underground*
*The author of the diary and the diary itself
are, of course, imaginary. Nevertheless it is clear
that such persons as the writer of these notes
not only may, but positively must, exist in our
society, when we consider the circumstances i
processed_text = raw_text.lower()
processed_text = re.sub(r'[^\x00-\x7f]',r'', processed_text)
print('corpus length:', len(processed_text))
chars = sorted(list(set(processed_text)))
print('total chars:', len(chars))
char_indices = dict((c, i) for i, c in enumerate(chars))
indices_char = dict((i, c) for i, c in enumerate(chars))
corpus length: 265582 total chars: 58
# cut the text in semi-redundant sequences of maxlen characters
maxlen = 40
step = 3
sentences = []
next_chars = []
for i in range(0, len(processed_text) - maxlen, step):
sentences.append(processed_text[i: i + maxlen])
next_chars.append(processed_text[i + maxlen])
print('nb sequences:', len(sentences))
nb sequences: 88514
sentences
["project gutenberg's notes from the under",
"ject gutenberg's notes from the undergro",
"t gutenberg's notes from the underground",
"utenberg's notes from the underground, b",
"nberg's notes from the underground, by f",
"rg's notes from the underground, by feod",
's notes from the underground, by feodor ',
'otes from the underground, by feodor dos',
's from the underground, by feodor dostoe',
'rom the underground, by feodor dostoevsk',
' the underground, by feodor dostoevsky\r\n',
'e underground, by feodor dostoevsky\r\n\r\nt',
'nderground, by feodor dostoevsky\r\n\r\nthis',
'rground, by feodor dostoevsky\r\n\r\nthis eb',
'ound, by feodor dostoevsky\r\n\r\nthis ebook',
'd, by feodor dostoevsky\r\n\r\nthis ebook is',
'by feodor dostoevsky\r\n\r\nthis ebook is fo',
'feodor dostoevsky\r\n\r\nthis ebook is for t',
'dor dostoevsky\r\n\r\nthis ebook is for the ',
' dostoevsky\r\n\r\nthis ebook is for the use',
'stoevsky\r\n\r\nthis ebook is for the use of',
'evsky\r\n\r\nthis ebook is for the use of an',
'ky\r\n\r\nthis ebook is for the use of anyon',
'\n\r\nthis ebook is for the use of anyone a',
'this ebook is for the use of anyone anyw',
's ebook is for the use of anyone anywher',
'book is for the use of anyone anywhere a',
'k is for the use of anyone anywhere at n',
's for the use of anyone anywhere at no c',
'or the use of anyone anywhere at no cost',
'the use of anyone anywhere at no cost an',
' use of anyone anywhere at no cost and w',
'e of anyone anywhere at no cost and with',
'f anyone anywhere at no cost and with\r\na',
'nyone anywhere at no cost and with\r\nalmo',
'ne anywhere at no cost and with\r\nalmost ',
'anywhere at no cost and with\r\nalmost no ',
'where at no cost and with\r\nalmost no res',
're at no cost and with\r\nalmost no restri',
'at no cost and with\r\nalmost no restricti',
'no cost and with\r\nalmost no restrictions',
'cost and with\r\nalmost no restrictions wh',
't and with\r\nalmost no restrictions whats',
'nd with\r\nalmost no restrictions whatsoev',
'with\r\nalmost no restrictions whatsoever.',
'h\r\nalmost no restrictions whatsoever. y',
'almost no restrictions whatsoever. you ',
'ost no restrictions whatsoever. you may',
' no restrictions whatsoever. you may co',
' restrictions whatsoever. you may copy ',
'strictions whatsoever. you may copy it,',
'ictions whatsoever. you may copy it, gi',
'ions whatsoever. you may copy it, give ',
's whatsoever. you may copy it, give it ',
'hatsoever. you may copy it, give it awa',
'soever. you may copy it, give it away o',
'ver. you may copy it, give it away or\r\n',
'. you may copy it, give it away or\r\nre-',
'you may copy it, give it away or\r\nre-use',
' may copy it, give it away or\r\nre-use it',
'y copy it, give it away or\r\nre-use it un',
'opy it, give it away or\r\nre-use it under',
' it, give it away or\r\nre-use it under th',
', give it away or\r\nre-use it under the t',
'ive it away or\r\nre-use it under the term',
' it away or\r\nre-use it under the terms o',
' away or\r\nre-use it under the terms of t',
'ay or\r\nre-use it under the terms of the ',
'or\r\nre-use it under the terms of the pro',
'\nre-use it under the terms of the projec',
'-use it under the terms of the project g',
'e it under the terms of the project gute',
't under the terms of the project gutenbe',
'nder the terms of the project gutenberg ',
'r the terms of the project gutenberg lic',
'he terms of the project gutenberg licens',
'terms of the project gutenberg license i',
'ms of the project gutenberg license incl',
'of the project gutenberg license include',
'the project gutenberg license included\r\n',
' project gutenberg license included\r\nwit',
'oject gutenberg license included\r\nwith t',
'ct gutenberg license included\r\nwith this',
'gutenberg license included\r\nwith this eb',
'enberg license included\r\nwith this ebook',
'erg license included\r\nwith this ebook or',
' license included\r\nwith this ebook or on',
'cense included\r\nwith this ebook or onlin',
'se included\r\nwith this ebook or online a',
'included\r\nwith this ebook or online at w',
'luded\r\nwith this ebook or online at www.',
'ed\r\nwith this ebook or online at www.gut',
'\nwith this ebook or online at www.gutenb',
'th this ebook or online at www.gutenberg',
'this ebook or online at www.gutenberg.ne',
's ebook or online at www.gutenberg.net\r\n',
'book or online at www.gutenberg.net\r\n\r\n\r',
'k or online at www.gutenberg.net\r\n\r\n\r\nti',
'r online at www.gutenberg.net\r\n\r\n\r\ntitle',
'nline at www.gutenberg.net\r\n\r\n\r\ntitle: n',
'ne at www.gutenberg.net\r\n\r\n\r\ntitle: note',
'at www.gutenberg.net\r\n\r\n\r\ntitle: notes f',
'www.gutenberg.net\r\n\r\n\r\ntitle: notes from',
'.gutenberg.net\r\n\r\n\r\ntitle: notes from th',
'tenberg.net\r\n\r\n\r\ntitle: notes from the u',
'berg.net\r\n\r\n\r\ntitle: notes from the unde',
'g.net\r\n\r\n\r\ntitle: notes from the undergr',
'et\r\n\r\n\r\ntitle: notes from the undergroun',
'\n\r\n\r\ntitle: notes from the underground\r\n',
'\r\ntitle: notes from the underground\r\n\r\na',
'itle: notes from the underground\r\n\r\nauth',
'e: notes from the underground\r\n\r\nauthor:',
'notes from the underground\r\n\r\nauthor: fe',
'es from the underground\r\n\r\nauthor: feodo',
'from the underground\r\n\r\nauthor: feodor d',
'm the underground\r\n\r\nauthor: feodor dost',
'he underground\r\n\r\nauthor: feodor dostoev',
'underground\r\n\r\nauthor: feodor dostoevsky',
'erground\r\n\r\nauthor: feodor dostoevsky\r\n\r',
'round\r\n\r\nauthor: feodor dostoevsky\r\n\r\npo',
'nd\r\n\r\nauthor: feodor dostoevsky\r\n\r\nposti',
'\n\r\nauthor: feodor dostoevsky\r\n\r\nposting ',
'author: feodor dostoevsky\r\n\r\nposting dat',
'hor: feodor dostoevsky\r\n\r\nposting date: ',
': feodor dostoevsky\r\n\r\nposting date: sep',
'eodor dostoevsky\r\n\r\nposting date: septem',
'or dostoevsky\r\n\r\nposting date: september',
'dostoevsky\r\n\r\nposting date: september 13',
'toevsky\r\n\r\nposting date: september 13, 2',
'vsky\r\n\r\nposting date: september 13, 2008',
'y\r\n\r\nposting date: september 13, 2008 [e',
'\r\nposting date: september 13, 2008 [eboo',
'osting date: september 13, 2008 [ebook #',
'ing date: september 13, 2008 [ebook #600',
' date: september 13, 2008 [ebook #600]\r\n',
'te: september 13, 2008 [ebook #600]\r\nrel',
' september 13, 2008 [ebook #600]\r\nreleas',
'ptember 13, 2008 [ebook #600]\r\nrelease d',
'mber 13, 2008 [ebook #600]\r\nrelease date',
'r 13, 2008 [ebook #600]\r\nrelease date: j',
'3, 2008 [ebook #600]\r\nrelease date: july',
'2008 [ebook #600]\r\nrelease date: july, 1',
'8 [ebook #600]\r\nrelease date: july, 1996',
'ebook #600]\r\nrelease date: july, 1996\r\n\r',
'ok #600]\r\nrelease date: july, 1996\r\n\r\nla',
'#600]\r\nrelease date: july, 1996\r\n\r\nlangu',
'0]\r\nrelease date: july, 1996\r\n\r\nlanguage',
'\nrelease date: july, 1996\r\n\r\nlanguage: e',
'lease date: july, 1996\r\n\r\nlanguage: engl',
'se date: july, 1996\r\n\r\nlanguage: english',
'date: july, 1996\r\n\r\nlanguage: english\r\n\r',
'e: july, 1996\r\n\r\nlanguage: english\r\n\r\n\r\n',
'july, 1996\r\n\r\nlanguage: english\r\n\r\n\r\n***',
'y, 1996\r\n\r\nlanguage: english\r\n\r\n\r\n*** st',
'1996\r\n\r\nlanguage: english\r\n\r\n\r\n*** start',
'6\r\n\r\nlanguage: english\r\n\r\n\r\n*** start of',
'\r\nlanguage: english\r\n\r\n\r\n*** start of th',
'anguage: english\r\n\r\n\r\n*** start of this ',
'uage: english\r\n\r\n\r\n*** start of this pro',
'e: english\r\n\r\n\r\n*** start of this projec',
'english\r\n\r\n\r\n*** start of this project g',
'lish\r\n\r\n\r\n*** start of this project gute',
'h\r\n\r\n\r\n*** start of this project gutenbe',
'\r\n\r\n*** start of this project gutenberg ',
'\n*** start of this project gutenberg ebo',
'* start of this project gutenberg ebook ',
'tart of this project gutenberg ebook not',
't of this project gutenberg ebook notes ',
'f this project gutenberg ebook notes fro',
'his project gutenberg ebook notes from t',
' project gutenberg ebook notes from the ',
'oject gutenberg ebook notes from the und',
'ct gutenberg ebook notes from the underg',
'gutenberg ebook notes from the undergrou',
'enberg ebook notes from the underground ',
'erg ebook notes from the underground ***',
' ebook notes from the underground ***\r\n\r',
'ook notes from the underground ***\r\n\r\n\r\n',
' notes from the underground ***\r\n\r\n\r\n\r\n\r',
'tes from the underground ***\r\n\r\n\r\n\r\n\r\npr',
' from the underground ***\r\n\r\n\r\n\r\n\r\nprodu',
'om the underground ***\r\n\r\n\r\n\r\n\r\nproduced',
'the underground ***\r\n\r\n\r\n\r\n\r\nproduced by',
' underground ***\r\n\r\n\r\n\r\n\r\nproduced by ju',
'derground ***\r\n\r\n\r\n\r\n\r\nproduced by judit',
'ground ***\r\n\r\n\r\n\r\n\r\nproduced by judith b',
'und ***\r\n\r\n\r\n\r\n\r\nproduced by judith boss',
' ***\r\n\r\n\r\n\r\n\r\nproduced by judith boss. ',
'*\r\n\r\n\r\n\r\n\r\nproduced by judith boss. htm',
'\r\n\r\n\r\n\r\nproduced by judith boss. html v',
'\n\r\n\r\nproduced by judith boss. html vers',
'\r\nproduced by judith boss. html version',
'roduced by judith boss. html version by',
'uced by judith boss. html version by al',
'd by judith boss. html version by al ha',
'y judith boss. html version by al haine',
'udith boss. html version by al haines.\r',
'th boss. html version by al haines.\r\n\r\n',
'boss. html version by al haines.\r\n\r\n\r\n\r',
's. html version by al haines.\r\n\r\n\r\n\r\n\r\n',
' html version by al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r',
'ml version by al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n',
'version by al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nn',
'sion by al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnote',
'n by al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes f',
'y al haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from',
'l haines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from th',
'aines.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from the u',
'es.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from the unde',
'\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from the undergr',
'\n\r\n\r\n\r\n\r\n\r\n\r\n\r\nnotes from the undergroun',
'\r\n\r\n\r\n\r\n\r\n\r\nnotes from the underground\r\n',
'\n\r\n\r\n\r\n\r\nnotes from the underground\r\n\r\nf',
'\r\n\r\n\r\nnotes from the underground\r\n\r\nfyod',
'\n\r\nnotes from the underground\r\n\r\nfyodor ',
'notes from the underground\r\n\r\nfyodor dos',
'es from the underground\r\n\r\nfyodor dostoy',
'from the underground\r\n\r\nfyodor dostoyevs',
'm the underground\r\n\r\nfyodor dostoyevsky\r',
'he underground\r\n\r\nfyodor dostoyevsky\r\n\r\n',
'underground\r\n\r\nfyodor dostoyevsky\r\n\r\n\r\n\r',
'erground\r\n\r\nfyodor dostoyevsky\r\n\r\n\r\n\r\n\r\n',
'round\r\n\r\nfyodor dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\np',
'nd\r\n\r\nfyodor dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart',
'\n\r\nfyodor dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r',
'fyodor dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\n',
'dor dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nund',
' dostoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nunderg',
'stoyevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nundergrou',
'yevsky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nunderground*',
'sky\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nunderground*\r\n\r',
'\r\n\r\n\r\n\r\n\r\n\r\npart i\r\n\r\nunderground*\r\n\r\n ',
'\n\r\n\r\n\r\n\r\npart i\r\n\r\nunderground*\r\n\r\n ',
'\r\n\r\n\r\npart i\r\n\r\nunderground*\r\n\r\n *th',
'\n\r\npart i\r\n\r\nunderground*\r\n\r\n *the a',
'part i\r\n\r\nunderground*\r\n\r\n *the auth',
't i\r\n\r\nunderground*\r\n\r\n *the author ',
'\r\n\r\nunderground*\r\n\r\n *the author of ',
'\nunderground*\r\n\r\n *the author of the',
'derground*\r\n\r\n *the author of the di',
'ground*\r\n\r\n *the author of the diary',
'und*\r\n\r\n *the author of the diary an',
'*\r\n\r\n *the author of the diary and t',
'\r\n *the author of the diary and the ',
' *the author of the diary and the dia',
' *the author of the diary and the diary ',
'he author of the diary and the diary its',
'author of the diary and the diary itself',
'hor of the diary and the diary itself\r\n ',
' of the diary and the diary itself\r\n ',
' the diary and the diary itself\r\n ar',
'e diary and the diary itself\r\n are, ',
'iary and the diary itself\r\n are, of ',
'y and the diary itself\r\n are, of cou',
'nd the diary itself\r\n are, of course',
'the diary itself\r\n are, of course, i',
' diary itself\r\n are, of course, imag',
'ary itself\r\n are, of course, imagina',
' itself\r\n are, of course, imaginary.',
'self\r\n are, of course, imaginary. n',
'f\r\n are, of course, imaginary. neve',
' are, of course, imaginary. neverth',
' are, of course, imaginary. neverthele',
're, of course, imaginary. nevertheless ',
' of course, imaginary. nevertheless it ',
' course, imaginary. nevertheless it is ',
'urse, imaginary. nevertheless it is cle',
'e, imaginary. nevertheless it is clear\r',
'imaginary. nevertheless it is clear\r\n ',
'ginary. nevertheless it is clear\r\n ',
'ary. nevertheless it is clear\r\n tha',
'. nevertheless it is clear\r\n that s',
'nevertheless it is clear\r\n that such',
'ertheless it is clear\r\n that such pe',
'heless it is clear\r\n that such perso',
'ess it is clear\r\n that such persons ',
' it is clear\r\n that such persons as ',
' is clear\r\n that such persons as the',
' clear\r\n that such persons as the wr',
'ear\r\n that such persons as the write',
'\r\n that such persons as the writer o',
' that such persons as the writer of t',
' that such persons as the writer of thes',
'at such persons as the writer of these n',
'such persons as the writer of these note',
'h persons as the writer of these notes\r\n',
'ersons as the writer of these notes\r\n ',
'ons as the writer of these notes\r\n n',
' as the writer of these notes\r\n not ',
' the writer of these notes\r\n not onl',
'e writer of these notes\r\n not only m',
'riter of these notes\r\n not only may,',
'er of these notes\r\n not only may, bu',
'of these notes\r\n not only may, but p',
'these notes\r\n not only may, but posi',
'se notes\r\n not only may, but positiv',
'notes\r\n not only may, but positively',
'es\r\n not only may, but positively mu',
'\n not only may, but positively must,',
' not only may, but positively must, ex',
'not only may, but positively must, exist',
' only may, but positively must, exist in',
'ly may, but positively must, exist in ou',
'may, but positively must, exist in our\r\n',
', but positively must, exist in our\r\n ',
'ut positively must, exist in our\r\n s',
'positively must, exist in our\r\n soci',
'itively must, exist in our\r\n society',
'vely must, exist in our\r\n society, w',
'y must, exist in our\r\n society, when',
'ust, exist in our\r\n society, when we',
', exist in our\r\n society, when we co',
'xist in our\r\n society, when we consi',
't in our\r\n society, when we consider',
'n our\r\n society, when we consider th',
'ur\r\n society, when we consider the c',
'\n society, when we consider the circ',
' society, when we consider the circums',
'society, when we consider the circumstan',
'iety, when we consider the circumstances',
'y, when we consider the circumstances in',
'when we consider the circumstances in\r\n ',
'n we consider the circumstances in\r\n ',
'e consider the circumstances in\r\n th',
'onsider the circumstances in\r\n the m',
'ider the circumstances in\r\n the mids',
'r the circumstances in\r\n the midst o',
'he circumstances in\r\n the midst of w',
'circumstances in\r\n the midst of whic',
'cumstances in\r\n the midst of which o',
'stances in\r\n the midst of which our ',
'nces in\r\n the midst of which our soc',
's in\r\n the midst of which our societ',
'n\r\n the midst of which our society i',
' the midst of which our society is f',
' the midst of which our society is form',
'he midst of which our society is formed.',
'midst of which our society is formed. i',
'st of which our society is formed. i ha',
'of which our society is formed. i have\r',
'which our society is formed. i have\r\n ',
'ch our society is formed. i have\r\n ',
'our society is formed. i have\r\n tri',
' society is formed. i have\r\n tried ',
'ciety is formed. i have\r\n tried to ',
'ty is formed. i have\r\n tried to exp',
'is formed. i have\r\n tried to expose',
'formed. i have\r\n tried to expose to',
'med. i have\r\n tried to expose to th',
'. i have\r\n tried to expose to the v',
'i have\r\n tried to expose to the view',
'ave\r\n tried to expose to the view of',
'\r\n tried to expose to the view of th',
' tried to expose to the view of the p',
' tried to expose to the view of the publ',
'ied to expose to the view of the public ',
' to expose to the view of the public mor',
' expose to the view of the public more\r\n',
'pose to the view of the public more\r\n ',
'e to the view of the public more\r\n d',
'o the view of the public more\r\n dist',
'he view of the public more\r\n distinc',
'view of the public more\r\n distinctly',
'w of the public more\r\n distinctly th',
'f the public more\r\n distinctly than ',
'he public more\r\n distinctly than is ',
'public more\r\n distinctly than is com',
'lic more\r\n distinctly than is common',
' more\r\n distinctly than is commonly ',
're\r\n distinctly than is commonly don',
'\n distinctly than is commonly done, ',
' distinctly than is commonly done, one',
'distinctly than is commonly done, one of',
'tinctly than is commonly done, one of th',
'ctly than is commonly done, one of the\r\n',
'y than is commonly done, one of the\r\n ',
'han is commonly done, one of the\r\n c',
' is commonly done, one of the\r\n char',
' commonly done, one of the\r\n charact',
'mmonly done, one of the\r\n characters',
'nly done, one of the\r\n characters of',
' done, one of the\r\n characters of th',
'ne, one of the\r\n characters of the r',
' one of the\r\n characters of the rece',
'e of the\r\n characters of the recent ',
'f the\r\n characters of the recent pas',
'he\r\n characters of the recent past. ',
'\n characters of the recent past. he',
' characters of the recent past. he is',
'characters of the recent past. he is on',
'racters of the recent past. he is one o',
'ters of the recent past. he is one of t',
's of the recent past. he is one of the\r',
'f the recent past. he is one of the\r\n ',
'he recent past. he is one of the\r\n ',
'recent past. he is one of the\r\n rep',
'ent past. he is one of the\r\n repres',
' past. he is one of the\r\n represent',
'st. he is one of the\r\n representati',
' he is one of the\r\n representatives',
'e is one of the\r\n representatives of',
's one of the\r\n representatives of a ',
'ne of the\r\n representatives of a gen',
'of the\r\n representatives of a genera',
'the\r\n representatives of a generatio',
'\r\n representatives of a generation s',
' representatives of a generation stil',
' representatives of a generation still l',
'presentatives of a generation still livi',
'sentatives of a generation still living.',
'tatives of a generation still living. i',
'ives of a generation still living. in t',
's of a generation still living. in this',
'f a generation still living. in this\r\n ',
' generation still living. in this\r\n ',
'neration still living. in this\r\n fr',
'ation still living. in this\r\n fragm',
'on still living. in this\r\n fragment',
'still living. in this\r\n fragment, e',
'll living. in this\r\n fragment, enti',
'living. in this\r\n fragment, entitle',
'ing. in this\r\n fragment, entitled "',
'. in this\r\n fragment, entitled "und',
'in this\r\n fragment, entitled "underg',
'this\r\n fragment, entitled "undergrou',
's\r\n fragment, entitled "underground,',
' fragment, entitled "underground," t',
' fragment, entitled "underground," this',
'ragment, entitled "underground," this pe',
'ment, entitled "underground," this perso',
't, entitled "underground," this person\r\n',
'entitled "underground," this person\r\n ',
'itled "underground," this person\r\n i',
'ed "underground," this person\r\n intr',
'"underground," this person\r\n introdu',
'derground," this person\r\n introduces',
'ground," this person\r\n introduces hi',
'und," this person\r\n introduces himse',
'," this person\r\n introduces himself ',
'this person\r\n introduces himself and',
's person\r\n introduces himself and hi',
'erson\r\n introduces himself and his v',
'on\r\n introduces himself and his view',
'\n introduces himself and his views, ',
' introduces himself and his views, and',
'introduces himself and his views, and, a',
'roduces himself and his views, and, as i',
'uces himself and his views, and, as it w',
's himself and his views, and, as it were',
'imself and his views, and, as it were,\r\n',
'elf and his views, and, as it were,\r\n ',
' and his views, and, as it were,\r\n t',
'd his views, and, as it were,\r\n trie',
'is views, and, as it were,\r\n tries t',
'views, and, as it were,\r\n tries to e',
'ws, and, as it were,\r\n tries to expl',
' and, as it were,\r\n tries to explain',
'd, as it were,\r\n tries to explain th',
'as it were,\r\n tries to explain the c',
'it were,\r\n tries to explain the caus',
'were,\r\n tries to explain the causes ',
'e,\r\n tries to explain the causes owi',
'\n tries to explain the causes owing ',
' tries to explain the causes owing to ',
'tries to explain the causes owing to whi',
'es to explain the causes owing to which ',
'to explain the causes owing to which he ',
'explain the causes owing to which he has',
'lain the causes owing to which he has\r\n ',
'n the causes owing to which he has\r\n ',
'he causes owing to which he has\r\n ma',
'causes owing to which he has\r\n made ',
'ses owing to which he has\r\n made his',
' owing to which he has\r\n made his ap',
'ing to which he has\r\n made his appea',
' to which he has\r\n made his appearan',
' which he has\r\n made his appearance ',
'ich he has\r\n made his appearance and',
' he has\r\n made his appearance and wa',
' has\r\n made his appearance and was b',
's\r\n made his appearance and was boun',
' made his appearance and was bound t',
' made his appearance and was bound to m',
'ade his appearance and was bound to make',
' his appearance and was bound to make hi',
's appearance and was bound to make his\r\n',
'ppearance and was bound to make his\r\n ',
'arance and was bound to make his\r\n a',
'nce and was bound to make his\r\n appe',
' and was bound to make his\r\n appeara',
'd was bound to make his\r\n appearance',
'as bound to make his\r\n appearance in',
'bound to make his\r\n appearance in ou',
'nd to make his\r\n appearance in our m',
'to make his\r\n appearance in our mids',
'make his\r\n appearance in our midst. ',
'e his\r\n appearance in our midst. in',
'is\r\n appearance in our midst. in th',
'\n appearance in our midst. in the s',
' appearance in our midst. in the seco',
'appearance in our midst. in the second ',
'earance in our midst. in the second fra',
'ance in our midst. in the second fragme',
'e in our midst. in the second fragment\r',
'n our midst. in the second fragment\r\n ',
'ur midst. in the second fragment\r\n ',
'midst. in the second fragment\r\n the',
'st. in the second fragment\r\n there ',
' in the second fragment\r\n there are',
'n the second fragment\r\n there are ad',
'he second fragment\r\n there are added',
'second fragment\r\n there are added th',
'ond fragment\r\n there are added the a',
' fragment\r\n there are added the actu',
'agment\r\n there are added the actual ',
'ent\r\n there are added the actual not',
'\r\n there are added the actual notes ',
' there are added the actual notes of ',
' there are added the actual notes of thi',
'ere are added the actual notes of this p',
' are added the actual notes of this pers',
'e added the actual notes of this person\r',
'dded the actual notes of this person\r\n ',
'd the actual notes of this person\r\n ',
'he actual notes of this person\r\n con',
'actual notes of this person\r\n concer',
'ual notes of this person\r\n concernin',
' notes of this person\r\n concerning c',
'tes of this person\r\n concerning cert',
' of this person\r\n concerning certain',
' this person\r\n concerning certain ev',
'is person\r\n concerning certain event',
'person\r\n concerning certain events i',
'son\r\n concerning certain events in h',
'\r\n concerning certain events in his ',
' concerning certain events in his lif',
' concerning certain events in his life.-',
'ncerning certain events in his life.--au',
'rning certain events in his life.--autho',
"ng certain events in his life.--author's",
"certain events in his life.--author's no",
"tain events in his life.--author's note.",
"n events in his life.--author's note.\r\n\r",
"vents in his life.--author's note.\r\n\r\n\r\n",
"ts in his life.--author's note.\r\n\r\n\r\n\r\ni",
"in his life.--author's note.\r\n\r\n\r\n\r\ni\r\n\r",
"his life.--author's note.\r\n\r\n\r\n\r\ni\r\n\r\ni ",
" life.--author's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am ",
"fe.--author's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am a s",
"--author's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick",
"uthor's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick ma",
"or's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick man..",
's note.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick man.... ',
'ote.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick man.... i ',
'.\r\n\r\n\r\n\r\ni\r\n\r\ni am a sick man.... i am ',
'\r\n\r\n\r\ni\r\n\r\ni am a sick man.... i am a s',
'\n\r\ni\r\n\r\ni am a sick man.... i am a spit',
'i\r\n\r\ni am a sick man.... i am a spitefu',
'\r\ni am a sick man.... i am a spiteful m',
' am a sick man.... i am a spiteful man.',
' a sick man.... i am a spiteful man. i',
'sick man.... i am a spiteful man. i am',
'k man.... i am a spiteful man. i am an',
'an.... i am a spiteful man. i am an un',
'... i am a spiteful man. i am an unatt',
' i am a spiteful man. i am an unattrac',
' am a spiteful man. i am an unattractiv',
' a spiteful man. i am an unattractive m',
'spiteful man. i am an unattractive man.',
'teful man. i am an unattractive man. i',
'ul man. i am an unattractive man. i\r\nb',
'man. i am an unattractive man. i\r\nbeli',
'. i am an unattractive man. i\r\nbelieve',
'i am an unattractive man. i\r\nbelieve my',
'm an unattractive man. i\r\nbelieve my li',
'n unattractive man. i\r\nbelieve my liver',
'nattractive man. i\r\nbelieve my liver is',
'tractive man. i\r\nbelieve my liver is di',
'ctive man. i\r\nbelieve my liver is disea',
've man. i\r\nbelieve my liver is diseased',
'man. i\r\nbelieve my liver is diseased. ',
'. i\r\nbelieve my liver is diseased. how',
'i\r\nbelieve my liver is diseased. howeve',
'believe my liver is diseased. however, ',
'ieve my liver is diseased. however, i k',
'e my liver is diseased. however, i know',
'y liver is diseased. however, i know no',
'iver is diseased. however, i know nothi',
'r is diseased. however, i know nothing ',
's diseased. however, i know nothing at ',
'iseased. however, i know nothing at all',
'ased. however, i know nothing at all ab',
'd. however, i know nothing at all about',
' however, i know nothing at all about my',
'wever, i know nothing at all about my\r\nd',
'er, i know nothing at all about my\r\ndise',
' i know nothing at all about my\r\ndisease',
'know nothing at all about my\r\ndisease, a',
'w nothing at all about my\r\ndisease, and ',
'othing at all about my\r\ndisease, and do ',
'ing at all about my\r\ndisease, and do not',
' at all about my\r\ndisease, and do not kn',
' all about my\r\ndisease, and do not know ',
'l about my\r\ndisease, and do not know for',
'bout my\r\ndisease, and do not know for ce',
't my\r\ndisease, and do not know for certa',
'y\r\ndisease, and do not know for certain ',
'disease, and do not know for certain wha',
'ease, and do not know for certain what a',
'e, and do not know for certain what ails',
'and do not know for certain what ails me',
' do not know for certain what ails me. ',
' not know for certain what ails me. i d',
"t know for certain what ails me. i don'",
"now for certain what ails me. i don't c",
" for certain what ails me. i don't cons",
"r certain what ails me. i don't consult",
"ertain what ails me. i don't consult a\r",
"ain what ails me. i don't consult a\r\ndo",
" what ails me. i don't consult a\r\ndocto",
"at ails me. i don't consult a\r\ndoctor f",
"ails me. i don't consult a\r\ndoctor for ",
"s me. i don't consult a\r\ndoctor for it,",
"e. i don't consult a\r\ndoctor for it, an",
" i don't consult a\r\ndoctor for it, and n",
"don't consult a\r\ndoctor for it, and neve",
"'t consult a\r\ndoctor for it, and never h",
'consult a\r\ndoctor for it, and never have',
'sult a\r\ndoctor for it, and never have, t',
't a\r\ndoctor for it, and never have, thou',
'\r\ndoctor for it, and never have, though ',
'octor for it, and never have, though i h',
'or for it, and never have, though i have',
'for it, and never have, though i have a ',
' it, and never have, though i have a res',
', and never have, though i have a respec',
'nd never have, though i have a respect f',
'never have, though i have a respect for ',
'er have, though i have a respect for med',
'have, though i have a respect for medici',
'e, though i have a respect for medicine ',
'though i have a respect for medicine and',
'ugh i have a respect for medicine and\r\nd',
' i have a respect for medicine and\r\ndoct',
'have a respect for medicine and\r\ndoctors',
'e a respect for medicine and\r\ndoctors. b',
' respect for medicine and\r\ndoctors. besi',
'spect for medicine and\r\ndoctors. besides',
'ct for medicine and\r\ndoctors. besides, i',
'for medicine and\r\ndoctors. besides, i am',
' medicine and\r\ndoctors. besides, i am ex',
'dicine and\r\ndoctors. besides, i am extre',
'ine and\r\ndoctors. besides, i am extremel',
' and\r\ndoctors. besides, i am extremely s',
'd\r\ndoctors. besides, i am extremely supe',
'doctors. besides, i am extremely superst',
'tors. besides, i am extremely superstiti',
's. besides, i am extremely superstitious',
'besides, i am extremely superstitious, s',
'ides, i am extremely superstitious, suff',
's, i am extremely superstitious, suffici',
'i am extremely superstitious, sufficient',
'm extremely superstitious, sufficiently ',
'xtremely superstitious, sufficiently so ',
'emely superstitious, sufficiently so to\r',
'ly superstitious, sufficiently so to\r\nre',
'superstitious, sufficiently so to\r\nrespe',
'erstitious, sufficiently so to\r\nrespect ',
'titious, sufficiently so to\r\nrespect med',
'ious, sufficiently so to\r\nrespect medici',
's, sufficiently so to\r\nrespect medicine,',
'sufficiently so to\r\nrespect medicine, an',
'ficiently so to\r\nrespect medicine, anywa',
'iently so to\r\nrespect medicine, anyway (',
'tly so to\r\nrespect medicine, anyway (i a',
' so to\r\nrespect medicine, anyway (i am w',
' to\r\nrespect medicine, anyway (i am well',
'\r\nrespect medicine, anyway (i am well-ed',
'espect medicine, anyway (i am well-educa',
'ect medicine, anyway (i am well-educated',
' medicine, anyway (i am well-educated en',
'dicine, anyway (i am well-educated enoug',
'ine, anyway (i am well-educated enough n',
', anyway (i am well-educated enough not ',
'nyway (i am well-educated enough not to ',
'ay (i am well-educated enough not to be\r',
'(i am well-educated enough not to be\r\nsu',
'am well-educated enough not to be\r\nsuper',
'well-educated enough not to be\r\nsupersti',
'l-educated enough not to be\r\nsuperstitio',
'ducated enough not to be\r\nsuperstitious,',
'ated enough not to be\r\nsuperstitious, bu',
'd enough not to be\r\nsuperstitious, but i',
'nough not to be\r\nsuperstitious, but i am',
'gh not to be\r\nsuperstitious, but i am su',
'not to be\r\nsuperstitious, but i am super',
' to be\r\nsuperstitious, but i am supersti',
' be\r\nsuperstitious, but i am superstitio',
'\r\nsuperstitious, but i am superstitious)',
'uperstitious, but i am superstitious). ',
'rstitious, but i am superstitious). no,',
'itious, but i am superstitious). no, i ',
'ous, but i am superstitious). no, i ref',
', but i am superstitious). no, i refuse',
'ut i am superstitious). no, i refuse to',
'i am superstitious). no, i refuse to co',
'm superstitious). no, i refuse to consu',
'uperstitious). no, i refuse to consult ',
'rstitious). no, i refuse to consult a\r\n',
'itious). no, i refuse to consult a\r\ndoc',
'ous). no, i refuse to consult a\r\ndoctor',
'). no, i refuse to consult a\r\ndoctor fr',
' no, i refuse to consult a\r\ndoctor from ',
', i refuse to consult a\r\ndoctor from spi',
' refuse to consult a\r\ndoctor from spite.',
'fuse to consult a\r\ndoctor from spite. t',
'e to consult a\r\ndoctor from spite. that',
'o consult a\r\ndoctor from spite. that yo',
'onsult a\r\ndoctor from spite. that you p',
'ult a\r\ndoctor from spite. that you prob',
' a\r\ndoctor from spite. that you probabl',
'\ndoctor from spite. that you probably w',
'ctor from spite. that you probably will',
'r from spite. that you probably will no',
'rom spite. that you probably will not u',
' spite. that you probably will not unde',
'ite. that you probably will not underst',
'. that you probably will not understand',
'that you probably will not understand. ',
't you probably will not understand. wel',
'ou probably will not understand. well, ',
'probably will not understand. well, i\r\n',
'bably will not understand. well, i\r\nund',
'ly will not understand. well, i\r\nunders',
'will not understand. well, i\r\nunderstan',
'l not understand. well, i\r\nunderstand i',
'ot understand. well, i\r\nunderstand it, ',
'understand. well, i\r\nunderstand it, tho',
'erstand. well, i\r\nunderstand it, though',
'tand. well, i\r\nunderstand it, though. ',
'd. well, i\r\nunderstand it, though. of ',
' well, i\r\nunderstand it, though. of cou',
'll, i\r\nunderstand it, though. of course',
' i\r\nunderstand it, though. of course, i',
'\nunderstand it, though. of course, i ca',
"derstand it, though. of course, i can't",
"stand it, though. of course, i can't ex",
"nd it, though. of course, i can't expla",
"it, though. of course, i can't explain ",
" though. of course, i can't explain who",
"ough. of course, i can't explain who it",
"h. of course, i can't explain who it is",
" of course, i can't explain who it is pr",
" course, i can't explain who it is preci",
"urse, i can't explain who it is precisel",
"e, i can't explain who it is precisely\r\n",
"i can't explain who it is precisely\r\ntha",
"an't explain who it is precisely\r\nthat i",
't explain who it is precisely\r\nthat i am',
'xplain who it is precisely\r\nthat i am mo',
'ain who it is precisely\r\nthat i am morti',
' who it is precisely\r\nthat i am mortifyi',
'o it is precisely\r\nthat i am mortifying ',
't is precisely\r\nthat i am mortifying in ',
's precisely\r\nthat i am mortifying in thi',
'recisely\r\nthat i am mortifying in this c',
'isely\r\nthat i am mortifying in this case',
'ly\r\nthat i am mortifying in this case by',
'\nthat i am mortifying in this case by my',
'at i am mortifying in this case by my sp',
'i am mortifying in this case by my spite',
'm mortifying in this case by my spite: i',
'ortifying in this case by my spite: i am',
'ifying in this case by my spite: i am pe',
'ing in this case by my spite: i am perfe',
' in this case by my spite: i am perfectl',
' this case by my spite: i am perfectly w',
'is case by my spite: i am perfectly well',
'case by my spite: i am perfectly well\r\na',
'e by my spite: i am perfectly well\r\nawar',
'y my spite: i am perfectly well\r\naware t',
'y spite: i am perfectly well\r\naware that',
'pite: i am perfectly well\r\naware that i ',
'e: i am perfectly well\r\naware that i can',
'i am perfectly well\r\naware that i cannot',
'm perfectly well\r\naware that i cannot "p',
'erfectly well\r\naware that i cannot "pay ',
'ectly well\r\naware that i cannot "pay out',
'ly well\r\naware that i cannot "pay out" t',
'well\r\naware that i cannot "pay out" the ',
'l\r\naware that i cannot "pay out" the doc',
'aware that i cannot "pay out" the doctor',
're that i cannot "pay out" the doctors b',
'that i cannot "pay out" the doctors by n',
't i cannot "pay out" the doctors by not ',
' cannot "pay out" the doctors by not con',
'nnot "pay out" the doctors by not consul',
't "pay out" the doctors by not consultin',
'pay out" the doctors by not consulting t',
' out" the doctors by not consulting them',
't" the doctors by not consulting them; i',
'the doctors by not consulting them; i\r\nk',
' doctors by not consulting them; i\r\nknow',
'ctors by not consulting them; i\r\nknow be',
'rs by not consulting them; i\r\nknow bette',
'by not consulting them; i\r\nknow better t',
'not consulting them; i\r\nknow better than',
' consulting them; i\r\nknow better than an',
'nsulting them; i\r\nknow better than anyon',
'lting them; i\r\nknow better than anyone t',
'ng them; i\r\nknow better than anyone that',
'them; i\r\nknow better than anyone that by',
'm; i\r\nknow better than anyone that by al',
'i\r\nknow better than anyone that by all t',
'know better than anyone that by all this',
'w better than anyone that by all this i ',
'etter than anyone that by all this i am ',
'er than anyone that by all this i am onl',
'than anyone that by all this i am only i',
'n anyone that by all this i am only inju',
'nyone that by all this i am only injurin',
'ne that by all this i am only injuring m',
'that by all this i am only injuring myse',
't by all this i am only injuring myself ',
'y all this i am only injuring myself and',
'll this i am only injuring myself and\r\nn',
'this i am only injuring myself and\r\nno o',
's i am only injuring myself and\r\nno one ',
' am only injuring myself and\r\nno one els',
' only injuring myself and\r\nno one else. ',
'ly injuring myself and\r\nno one else. bu',
'injuring myself and\r\nno one else. but s',
'uring myself and\r\nno one else. but stil',
'ng myself and\r\nno one else. but still, ',
'myself and\r\nno one else. but still, if ',
'elf and\r\nno one else. but still, if i d',
" and\r\nno one else. but still, if i don'",
"d\r\nno one else. but still, if i don't c",
"no one else. but still, if i don't cons",
"one else. but still, if i don't consult",
" else. but still, if i don't consult a ",
"se. but still, if i don't consult a doc",
" but still, if i don't consult a doctor",
"ut still, if i don't consult a doctor it",
"still, if i don't consult a doctor it is",
"ll, if i don't consult a doctor it is fr",
" if i don't consult a doctor it is from ",
" i don't consult a doctor it is from spi",
"don't consult a doctor it is from spite.",
"'t consult a doctor it is from spite.\r\nm",
'consult a doctor it is from spite.\r\nmy l',
'sult a doctor it is from spite.\r\nmy live',
't a doctor it is from spite.\r\nmy liver i',
' doctor it is from spite.\r\nmy liver is b',
'ctor it is from spite.\r\nmy liver is bad,',
'r it is from spite.\r\nmy liver is bad, we',
't is from spite.\r\nmy liver is bad, well-',
's from spite.\r\nmy liver is bad, well--le',
'rom spite.\r\nmy liver is bad, well--let i',
' spite.\r\nmy liver is bad, well--let it g',
'ite.\r\nmy liver is bad, well--let it get ',
'.\r\nmy liver is bad, well--let it get wor',
'my liver is bad, well--let it get worse!',
'liver is bad, well--let it get worse!\r\n\r',
'er is bad, well--let it get worse!\r\n\r\ni ',
'is bad, well--let it get worse!\r\n\r\ni hav',
'bad, well--let it get worse!\r\n\r\ni have b',
', well--let it get worse!\r\n\r\ni have been',
'ell--let it get worse!\r\n\r\ni have been go',
'--let it get worse!\r\n\r\ni have been going',
'et it get worse!\r\n\r\ni have been going on',
'it get worse!\r\n\r\ni have been going on li',
'get worse!\r\n\r\ni have been going on like ',
' worse!\r\n\r\ni have been going on like tha',
'rse!\r\n\r\ni have been going on like that f',
'!\r\n\r\ni have been going on like that for ',
'\r\ni have been going on like that for a l',
' have been going on like that for a long',
've been going on like that for a long ti',
'been going on like that for a long time-',
'n going on like that for a long time--tw',
'oing on like that for a long time--twent',
'g on like that for a long time--twenty y',
'n like that for a long time--twenty year',
'ike that for a long time--twenty years. ',
' that for a long time--twenty years. no',
'at for a long time--twenty years. now i',
'for a long time--twenty years. now i am',
' a long time--twenty years. now i am\r\nf',
'long time--twenty years. now i am\r\nfort',
'g time--twenty years. now i am\r\nforty. ',
'ime--twenty years. now i am\r\nforty. i ',
'--twenty years. now i am\r\nforty. i use',
'wenty years. now i am\r\nforty. i used t',
'ty years. now i am\r\nforty. i used to b',
'years. now i am\r\nforty. i used to be i',
'rs. now i am\r\nforty. i used to be in t',
' now i am\r\nforty. i used to be in the ',
'ow i am\r\nforty. i used to be in the gov',
'i am\r\nforty. i used to be in the govern',
'm\r\nforty. i used to be in the governmen',
'forty. i used to be in the government s',
'ty. i used to be in the government serv',
' i used to be in the government service',
' used to be in the government service, b',
'ed to be in the government service, but ',
'to be in the government service, but am ',
'be in the government service, but am no ',
'in the government service, but am no lon',
'the government service, but am no longer',
' government service, but am no longer. ',
'vernment service, but am no longer. i\r\n',
'nment service, but am no longer. i\r\nwas',
'nt service, but am no longer. i\r\nwas a ',
'service, but am no longer. i\r\nwas a spi',
'vice, but am no longer. i\r\nwas a spitef',
'e, but am no longer. i\r\nwas a spiteful ',
'but am no longer. i\r\nwas a spiteful off',
' am no longer. i\r\nwas a spiteful offici',
' no longer. i\r\nwas a spiteful official.',
' longer. i\r\nwas a spiteful official. i',
'nger. i\r\nwas a spiteful official. i wa',
'r. i\r\nwas a spiteful official. i was r',
' i\r\nwas a spiteful official. i was rude',
'\nwas a spiteful official. i was rude an',
's a spiteful official. i was rude and t',
' spiteful official. i was rude and took',
'iteful official. i was rude and took pl',
'ful official. i was rude and took pleas',
' official. i was rude and took pleasure',
'ficial. i was rude and took pleasure in',
'ial. i was rude and took pleasure in be',
'. i was rude and took pleasure in being',
'i was rude and took pleasure in being so',
'as rude and took pleasure in being so. ',
'rude and took pleasure in being so. i\r\n',
'e and took pleasure in being so. i\r\ndid',
'nd took pleasure in being so. i\r\ndid no',
'took pleasure in being so. i\r\ndid not t',
'k pleasure in being so. i\r\ndid not take',
'leasure in being so. i\r\ndid not take br',
'sure in being so. i\r\ndid not take bribe',
'e in being so. i\r\ndid not take bribes, ',
'n being so. i\r\ndid not take bribes, you',
'eing so. i\r\ndid not take bribes, you se',
'g so. i\r\ndid not take bribes, you see, ',
'o. i\r\ndid not take bribes, you see, so ',
' i\r\ndid not take bribes, you see, so i w',
'\ndid not take bribes, you see, so i was ',
'd not take bribes, you see, so i was bou',
'ot take bribes, you see, so i was bound ',
'take bribes, you see, so i was bound to ',
'e bribes, you see, so i was bound to fin',
'ribes, you see, so i was bound to find a',
'es, you see, so i was bound to find a re',
' you see, so i was bound to find a recom',
'u see, so i was bound to find a recompen',
'ee, so i was bound to find a recompense ',
' so i was bound to find a recompense in\r',
' i was bound to find a recompense in\r\nth',
'was bound to find a recompense in\r\nthat,',
' bound to find a recompense in\r\nthat, at',
'und to find a recompense in\r\nthat, at le',
' to find a recompense in\r\nthat, at least',
' find a recompense in\r\nthat, at least. ',
'nd a recompense in\r\nthat, at least. (a ',
'a recompense in\r\nthat, at least. (a poo',
'ecompense in\r\nthat, at least. (a poor j',
'mpense in\r\nthat, at least. (a poor jest',
'nse in\r\nthat, at least. (a poor jest, b',
' in\r\nthat, at least. (a poor jest, but ',
'\r\nthat, at least. (a poor jest, but i w',
'hat, at least. (a poor jest, but i will',
', at least. (a poor jest, but i will no',
't least. (a poor jest, but i will not s',
'east. (a poor jest, but i will not scra',
't. (a poor jest, but i will not scratch',
' (a poor jest, but i will not scratch it',
' poor jest, but i will not scratch it ou',
'or jest, but i will not scratch it out. ',
'jest, but i will not scratch it out. i ',
't, but i will not scratch it out. i wro',
'but i will not scratch it out. i wrote\r',
' i will not scratch it out. i wrote\r\nit',
'will not scratch it out. i wrote\r\nit th',
'l not scratch it out. i wrote\r\nit think',
'ot scratch it out. i wrote\r\nit thinking',
'scratch it out. i wrote\r\nit thinking it',
'atch it out. i wrote\r\nit thinking it wo',
'h it out. i wrote\r\nit thinking it would',
't out. i wrote\r\nit thinking it would so',
'ut. i wrote\r\nit thinking it would sound',
' i wrote\r\nit thinking it would sound ve',
' wrote\r\nit thinking it would sound very ',
'ote\r\nit thinking it would sound very wit',
'\r\nit thinking it would sound very witty;',
't thinking it would sound very witty; bu',
'hinking it would sound very witty; but n',
'king it would sound very witty; but now ',
'g it would sound very witty; but now tha',
...]
print('Vectorization...')
x = np.zeros((len(sentences), maxlen, len(chars)), dtype=np.bool)
y = np.zeros((len(sentences), len(chars)), dtype=np.bool)
for i, sentence in enumerate(sentences):
for t, char in enumerate(sentence):
x[i, t, char_indices[char]] = 1
y[i, char_indices[next_chars[i]]] = 1
Vectorization...
x.shape
(88514, 40, 58)
y.shape
(88514, 58)
y[0:10]
array([[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, True, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, True, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, True, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, True, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
True, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, True, False, False, False, False, False, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, True,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, True, False, False,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, True,
False, False, False, False],
[False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, False, False, False, False, False, False, False,
False, False, True, False]])
# build the model: a single LSTM
print('Build model...')
model = Sequential()
model.add(LSTM(128, input_shape=(maxlen, len(chars))))
model.add(Dense(len(chars), activation='softmax'))
optimizer = RMSprop(lr=0.01)
model.compile(loss='categorical_crossentropy', optimizer=optimizer)
Build model...
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= lstm (LSTM) (None, 128) 95744 _________________________________________________________________ dense (Dense) (None, 58) 7482 ================================================================= Total params: 103,226 Trainable params: 103,226 Non-trainable params: 0 _________________________________________________________________
def sample(preds, temperature=1.0):
# helper function to sample an index from a probability array
preds = np.asarray(preds).astype('float64')
preds = np.log(preds) / temperature
exp_preds = np.exp(preds)
preds = exp_preds / np.sum(exp_preds)
probas = np.random.multinomial(1, preds, 1)
return np.argmax(probas)
def on_epoch_end(epoch, _):
# Function invoked at end of each epoch. Prints generated text.
print("******************************************************")
print('----- Generating text after Epoch: %d' % epoch)
start_index = random.randint(0, len(processed_text) - maxlen - 1)
for temperature in [0.2, 0.5, 1.0, 1.2]:
print('----- temperature:', temperature)
generated = ''
sentence = processed_text[start_index: start_index + maxlen]
generated += sentence
print('----- Generating with seed: "' + sentence + '"')
sys.stdout.write(generated)
for i in range(400):
x_pred = np.zeros((1, maxlen, len(chars)))
for t, char in enumerate(sentence):
x_pred[0, t, char_indices[char]] = 1.
preds = model.predict(x_pred, verbose=0)[0]
next_index = sample(preds, temperature)
next_char = indices_char[next_index]
generated += next_char
sentence = sentence[1:] + next_char
sys.stdout.write(next_char)
sys.stdout.flush()
print()
# Ignore useless W0819 warnings generated by TensorFlow 2.0. Hopefully can remove this ignore in the future.
# See https://github.com/tensorflow/tensorflow/issues/31308
import logging, os
logging.disable(logging.WARNING)
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "3"
# Fit the model
print_callback = LambdaCallback(on_epoch_end=on_epoch_end)
model.fit(x, y,
batch_size=128,
epochs=60,
callbacks=[print_callback])
Epoch 1/60 692/692 [==============================] - ETA: 0s - loss: 2.1576****************************************************** ----- Generating text after Epoch: 0 ----- temperature: 0.2 ----- Generating with seed: "cried. "but i will make up for it or p" cried. "but i will make up for it or pored the stine the derent of the dook and the stand to the store and the doon the sare the dong the dened to the dook the sare the stare the street of the seres at all the sall the deared the derent of the street in i am in the derent to the stan to the street the stine the sered to the dran that is all the stan the stare the dook the forest of the stan stand of the dook all the stand the streed t ----- temperature: 0.5 ----- Generating with seed: "cried. "but i will make up for it or p" cried. "but i he know conse the lake the s ----- temperature: 1.0 ----- Generating with seed: "cried. "but i will make up for it or p" cried. "but i "yes kle the lafine as the prosies netc ----- temperature: 1.2 ----- Generating with seed: "cried. "but i will make up for it or p" cried. "but i anp, ! 692/692 [==============================] - 155s 224ms/step - loss: 2.1576 Epoch 2/60 572/692 [=======================>......] - ETA: 13s - loss: 1.7386