Published by Pearson (March 15, 2019) © 2019
Paul Deitel | Harvey DeitelProduct Information
The professional programmer’s Deitel® guide to Python® with introductory artificial intelligence case studies
Written for programmers with a background in another high-level language, Python for Programmers uses hands-on instruction to teach today’s most compelling, leading-edge computing technologies and programming in Python–one of the world’s most popular and fastest-growing languages. Please read the Table of Contents diagram inside the front cover and the Preface for more details.
In the context of 500+, real-world examples ranging from individual snippets to 40 large scripts and full implementation case studies, you’ll use the interactive IPython interpreter with code in Jupyter Notebooks to quickly master the latest Python coding idioms. After covering Python Chapters 1-5 and a few key parts of Chapters 6-7, you’ll be able to handle significant portions of the hands-on introductory AI case studies in Chapters 11-16, which are loaded with cool, powerful, contemporary examples. These include natural language processing, data mining Twitter® for sentiment analysis, cognitive computing with IBM® Watson™, supervised machine learning with classification and regression, unsupervised machine learning with clustering, computer vision through deep learning and convolutional neural networks, deep learning with recurrent neural networks, big data with Hadoop®, Spark™ and NoSQL databases, the Internet of Things and more. You’ll also work directly or indirectly with cloud-based services, including Twitter, Google Translate™, IBM Watson, Microsoft® Azure®, OpenMapQuest, PubNub and more.
Features
Register your product for convenient access to downloads, updates, and/or corrections as they become available. See inside book for more information.
Written for programmers with a background in another high-level language, Python for Programmers uses hands-on instruction to teach today’s most compelling, leading-edge computing technologies and programming in Python–one of the world’s most popular and fastest-growing languages. Please read the Table of Contents diagram inside the front cover and the Preface for more details.
In the context of 500+, real-world examples ranging from individual snippets to 40 large scripts and full implementation case studies, you’ll use the interactive IPython interpreter with code in Jupyter Notebooks to quickly master the latest Python coding idioms. After covering Python Chapters 1-5 and a few key parts of Chapters 6-7, you’ll be able to handle significant portions of the hands-on introductory AI case studies in Chapters 11-16, which are loaded with cool, powerful, contemporary examples. These include natural language processing, data mining Twitter® for sentiment analysis, cognitive computing with IBM® Watson™, supervised machine learning with classification and regression, unsupervised machine learning with clustering, computer vision through deep learning and convolutional neural networks, deep learning with recurrent neural networks, big data with Hadoop®, Spark™ and NoSQL databases, the Internet of Things and more. You’ll also work directly or indirectly with cloud-based services, including Twitter, Google Translate™, IBM Watson, Microsoft® Azure®, OpenMapQuest, PubNub and more.
Features
- 500+ hands-on, real-world, live-code examples from snippets to case studies
- IPython + code in Jupyter® Notebooks
- Library-focused: Uses Python Standard Library and data science libraries to accomplish significant tasks with minimal code
- Rich Python coverage: Control statements, functions, strings, files, JSON serialization, CSV, exceptions
- Procedural, functional-style and object-oriented programming
- Collections: Lists, tuples, dictionaries, sets, NumPy arrays, pandas Series & DataFrames
- Static, dynamic and interactive visualizations
- Data experiences with real-world datasets and data sources
- Intro to Data Science sections: AI, basic stats, simulation, animation, random variables, data wrangling, regression
- AI, big data and cloud data science case studies: NLP, data mining Twitter®, IBM® Watson™, machine learning, deep learning, computer vision, Hadoop®, Spark™, NoSQL, IoT
- Open-source libraries: NumPy, pandas, Matplotlib, Seaborn, Folium, SciPy, NLTK, TextBlob, spaCy, Textatistic, Tweepy, scikit-learn®, Keras and more
Accompanying code examples are available here: http://ptgmedia.pearsoncmg.com/imprint_downloads/informit/bookreg/9780135224335/9780135224335_examples.zip.
Preface xvii
Before You Begin xxxiii
Chapter 1: Introduction to Computers and Python 1
1.1 Introduction 2
1.2 A Quick Review of Object Technology Basics 3
1.3 Python 5
1.4 It’s the Libraries! 7
1.5 Test-Drives: Using IPython and Jupyter Notebooks 9
1.6 The Cloud and the Internet of Things 16
1.7 How Big Is Big Data? 17
1.8 Case Study–A Big-Data Mobile Application 24
1.9 Intro to Data Science: Artificial Intelligence–at the Intersection of CS and Data Science 26
1.10 Wrap-Up 29
Chapter 2: Introduction to Python Programming 31
2.1 Introduction 32
2.2 Variables and Assignment Statements 32
2.3 Arithmetic 33
2.4 Function print and an Intro to Single- and Double-Quoted Strings 36
2.5 Triple-Quoted Strings 38
2.6 Getting Input from the User 39
2.7 Decision Making: The if Statement and Comparison Operators 41
2.8 Objects and Dynamic Typing 45
2.9 Intro to Data Science: Basic Descriptive Statistics 46
2.10 Wrap-Up 48
Chapter 3: Control Statements 49
3.1 Introduction 50
3.2 Control Statements 50
3.3 if Statement 51
3.4 if…else and if…elif…else Statements 52
3.5 while Statement 55
3.6 for Statement 55
3.7 Augmented Assignments 57
3.8 Sequence-Controlled Iteration; Formatted Strings 58
3.9 Sentinel-Controlled Iteration 59
3.10 Built-In Function range: A Deeper Look 60
3.11 Using Type Decimal for Monetary Amounts 61
3.12 break and continue Statements 64
3.13 Boolean Operators and, or and not 65
3.14 Intro to Data Science: Measures of Central Tendency–Mean, Median and Mode 67
3.15 Wrap-Up 69
Chapter 4: Functions 71
4.1 Introduction 72
4.2 Defining Functions 72
4.3 Functions with Multiple Parameters 75
4.4 Random-Number Generation 76
4.5 Case Study: A Game of Chance 78
4.6 Python Standard Library 81
4.7 math Module Functions 82
4.8 Using IPython Tab Completion for Discovery 83
4.9 Default Parameter Values 85
4.10 Keyword Arguments 85
4.11 Arbitrary Argument Lists 86
4.12 Methods: Functions That Belong to Objects 87
4.13 Scope Rules 87
4.14 import: A Deeper Look 89
4.15 Passing Arguments to Functions: A Deeper Look 90
4.16 Recursion 93
4.17 Functional-Style Programming 95
4.18 Intro to Data Science: Measures of Dispersion 97
4.19 Wrap-Up 98
Chapter 5: Sequences: Lists and Tuples 101
5.1 Introduction 102
5.2 Lists 102
5.3 Tuples 106
5.4 Unpacking Sequences 108
5.5 Sequence Slicing 110
5.6 del Statement 112
5.7 Passing Lists to Functions 113
5.8 Sorting Lists 115
5.9 Searching Sequences 116
5.10 Other List Methods 117
5.11 Simulating Stacks with Lists 119
5.12 List Comprehensions 120
5.13 Generator Expressions 121
5.14 Filter, Map and Reduce 122
5.15 Other Sequence Processing Functions 124
5.16 Two-Dimensional Lists 126
5.17 Intro to Data Science: Simulation and Static Visualizations 128
5.18 Wrap-Up 135
Chapter 6: Dictionaries and Sets 137
6.1 Introduction 138
6.2 Dictionaries 138
6.3 Sets 147
6.4 Intro to Data Science: Dynamic Visualizations 152
6.5 Wrap-Up 158
Chapter 7: Array-Oriented Programming with NumPy 159
7.1 Introduction 160
7.2 Creating arrays from Existing Data 160
7.3 array Attributes 161
7.4 Filling arrays with Specific Values 163
7.5 Creating arrays from Ranges 164
7.6 List vs. array Performance: Introducing %timeit 165
7.7 array Operators 167
7.8 NumPy Calculation Methods 169
7.9 Universal Functions 170
7.10 Indexing and Slicing 171
7.11 Views: Shallow Copies 173
7.12 Deep Copies 174
7.13 Reshaping and Transposing 175
7.14 Intro to Data Science: pandas Series and DataFrames 177
7.15 Wrap-Up 189
Chapter 8: Strings: A Deeper Look 191
8.1 Introduction 192
8.2 Formatting Strings 193
8.3 Concatenating and Repeating Strings 196
8.4 Stripping Whitespace from Strings 197
8.5 Changing Character Case 197
8.6 Comparison Operators for Strings 198
8.7 Searching for Substrings 198
8.8 Replacing Substrings 199
8.9 Splitting and Joining Strings 200
8.10 Characters and Character-Testing Methods 202
8.11 Raw Strings 203
8.12 Introduction to Regular Expressions 203
8.13 Intro to Data Science: Pandas, Regular Expressions and Data Munging 210
8.14 Wrap-Up 214
Chapter 9: Files and Exceptions 217
9.1 Introduction 218
9.2 Files 219
9.3 Text-File Processing 219
9.4 Updating Text Files 222
9.5 Serialization with JSON 223
9.6 Focus on Security: pickle Serialization and Deserialization 226
9.7 Additional Notes Regarding Files 226
9.8 Handling Exceptions 227
9.9 finally Clause 231
9.10 Explicitly Raising an Exception 233
9.11 (Optional) Stack Unwinding and Tracebacks 233
9.12 Intro to Data Science: Working with CSV Files 235
9.13 Wrap-Up 241
Chapter 10: Object-Oriented Programming 243
10.1 Introduction 244
10.2 Custom Class Account 246
10.3 Controlling Access to Attributes 249
10.4 Properties for Data Access 250
10.5 Simulating “Private” Attributes 256
10.6 Case Study: Card Shuffling and Dealing Simulation 258
10.7 Inheritance: Base Classes and Subclasses 266
10.8 Building an Inheritance Hierarchy; Introducing Polymorphism 267
10.9 Duck Typing and Polymorphism 275
10.10 Operator Overloading 276
10.11 Exception Class Hierarchy and Custom Exceptions 279
10.12 Named Tuples 280
10.13 A Brief Intro to Python 3.7’s New Data Classes 281
10.14 Unit Testing with Docstrings and doctest 287
10.15 Namespaces and Scopes 290
10.16 Intro to Data Science: Time Series and Simple Linear Regression 293
10.17 Wrap-Up 301
Chapter 11: Natural Language Processing (NLP) 303
11.1 Introduction 304
11.2 TextBlob 305
11.3 Visualizing Word Frequencies with Bar Charts and Word Clouds 319
11.4 Readability Assessment with Textatistic 324
11.5 Named Entity Recognition with spaCy 326
11.6 Similarity Detection with spaCy 327
11.7 Other NLP Libraries and Tools 328
11.8 Machine Learning and Deep Learning Natural Language Applications 328
11.9 Natural Language Datasets 329
11.10 Wrap-Up 330
Chapter 12: Data Mining Twitter 331
12.1 Introduction 332
12.2 Overview of the Twitter APIs 334
12.3 Creating a Twitter Account 335
12.4 Getting Twitter Credentials–Creating an App 335
12.5 What’s in a Tweet? 337
12.6 Tweepy 340
12.7 Authenticating with Twitter Via Tweepy 341
12.8 Getting Information About a Twitter Account 342
12.9 Introduction to Tweepy Cursors: Getting an Account’s Followers and Friends 344
12.10 Searching Recent Tweets 347
12.11 Spotting Trends: Twitter Trends API 349
12.12 Cleaning/Preprocessing Tweets for Analysis 353
12.13 Twitter Streaming API 354
12.14 Tweet Sentiment Analysis 359
12.15 Geocoding and Mapping 362
12.16 Ways to Store Tweets 370
12.17 Twitter and Time Series 370
12.18 Wrap-Up 371
Chapter 13: IBM Watson and Cognitive Computing 373
13.1 Introduction: IBM Watson and Cognitive Computing 374
13.2 IBM Cloud Account and Cloud Console 375
13.3 Watson Services 376
13.4 Additional Services and Tools 379
13.5 Watson Developer Cloud Python SDK 381
13.6 Case Study: Traveler’s Companion Translation App 381
13.7 Watson Resources 394
13.8 Wrap-Up 395
Chapter 14: Machine Learning: Classification, Regression and Clustering 397
14.1 Introduction to Machine Learning 398
14.2 Case Study: Classification with k-Nearest Neighbors and the Digits Dataset, Part 1 403
14.3 Case Study: Classification with k-Nearest Neighbors and the Digits Dataset, Part 2 413
14.4 Case Study: Time Series and Simple Linear Regression 420
14.5 Case Study: Multiple Linear Regression with the California Housing Dataset 425
14.6 Case Study: Unsupervised Machine Learning, Part 1–Dimensionality Reduction 438
14.7 Case Study: Unsupervised Machine Learning, Part 2–k-Means Clustering 442
14.8 Wrap-Up 455
Chapter 15: Deep Learning 457
15.1 Introduction 458
15.2 Keras Built-In Datasets 461
15.3 Custom Anaconda Environments 462
15.4 Neural Networks 463
15.5 Tensors 465
15.6 Convolutional Neural Networks for Vision; Multi-Classification with the MNIST Dataset 467
15.7 Visualizing Neural Network Training with TensorBoard 486
15.8 ConvnetJS: Browser-Based Deep-Learning Training and Visualization 489
15.9 Recurrent Neural Networks for Sequences; Sentiment Analysis with the IMDb Dataset 489
15.10 Tuning Deep Learning Models 497
15.11 Convnet Models Pretrained on ImageNet 498
15.12 Wrap-Up 499
Chapter 16: Big Data: Hadoop, Spark, NoSQL and IoT 501
16.1 Introduction 502
16.2 Relational Databases and Structured Query Language (SQL) 506
16.3 NoSQL and NewSQL Big-Data Databases: A Brief Tour 517
16.4 Case Study: A MongoDB JSON Document Database 520
16.5 Hadoop 530
16.6 Spark 541
16.7 Spark Streaming: Counting Twitter Hashtags Using the pyspark-notebook Docker Stack 551
16.8 Internet of Things and Dashboards 560
16.9 Wrap-Up 571
Index 573
Before You Begin xxxiii
Chapter 1: Introduction to Computers and Python 1
1.1 Introduction 2
1.2 A Quick Review of Object Technology Basics 3
1.3 Python 5
1.4 It’s the Libraries! 7
1.5 Test-Drives: Using IPython and Jupyter Notebooks 9
1.6 The Cloud and the Internet of Things 16
1.7 How Big Is Big Data? 17
1.8 Case Study–A Big-Data Mobile Application 24
1.9 Intro to Data Science: Artificial Intelligence–at the Intersection of CS and Data Science 26
1.10 Wrap-Up 29
Chapter 2: Introduction to Python Programming 31
2.1 Introduction 32
2.2 Variables and Assignment Statements 32
2.3 Arithmetic 33
2.4 Function print and an Intro to Single- and Double-Quoted Strings 36
2.5 Triple-Quoted Strings 38
2.6 Getting Input from the User 39
2.7 Decision Making: The if Statement and Comparison Operators 41
2.8 Objects and Dynamic Typing 45
2.9 Intro to Data Science: Basic Descriptive Statistics 46
2.10 Wrap-Up 48
Chapter 3: Control Statements 49
3.1 Introduction 50
3.2 Control Statements 50
3.3 if Statement 51
3.4 if…else and if…elif…else Statements 52
3.5 while Statement 55
3.6 for Statement 55
3.7 Augmented Assignments 57
3.8 Sequence-Controlled Iteration; Formatted Strings 58
3.9 Sentinel-Controlled Iteration 59
3.10 Built-In Function range: A Deeper Look 60
3.11 Using Type Decimal for Monetary Amounts 61
3.12 break and continue Statements 64
3.13 Boolean Operators and, or and not 65
3.14 Intro to Data Science: Measures of Central Tendency–Mean, Median and Mode 67
3.15 Wrap-Up 69
Chapter 4: Functions 71
4.1 Introduction 72
4.2 Defining Functions 72
4.3 Functions with Multiple Parameters 75
4.4 Random-Number Generation 76
4.5 Case Study: A Game of Chance 78
4.6 Python Standard Library 81
4.7 math Module Functions 82
4.8 Using IPython Tab Completion for Discovery 83
4.9 Default Parameter Values 85
4.10 Keyword Arguments 85
4.11 Arbitrary Argument Lists 86
4.12 Methods: Functions That Belong to Objects 87
4.13 Scope Rules 87
4.14 import: A Deeper Look 89
4.15 Passing Arguments to Functions: A Deeper Look 90
4.16 Recursion 93
4.17 Functional-Style Programming 95
4.18 Intro to Data Science: Measures of Dispersion 97
4.19 Wrap-Up 98
Chapter 5: Sequences: Lists and Tuples 101
5.1 Introduction 102
5.2 Lists 102
5.3 Tuples 106
5.4 Unpacking Sequences 108
5.5 Sequence Slicing 110
5.6 del Statement 112
5.7 Passing Lists to Functions 113
5.8 Sorting Lists 115
5.9 Searching Sequences 116
5.10 Other List Methods 117
5.11 Simulating Stacks with Lists 119
5.12 List Comprehensions 120
5.13 Generator Expressions 121
5.14 Filter, Map and Reduce 122
5.15 Other Sequence Processing Functions 124
5.16 Two-Dimensional Lists 126
5.17 Intro to Data Science: Simulation and Static Visualizations 128
5.18 Wrap-Up 135
Chapter 6: Dictionaries and Sets 137
6.1 Introduction 138
6.2 Dictionaries 138
6.3 Sets 147
6.4 Intro to Data Science: Dynamic Visualizations 152
6.5 Wrap-Up 158
Chapter 7: Array-Oriented Programming with NumPy 159
7.1 Introduction 160
7.2 Creating arrays from Existing Data 160
7.3 array Attributes 161
7.4 Filling arrays with Specific Values 163
7.5 Creating arrays from Ranges 164
7.6 List vs. array Performance: Introducing %timeit 165
7.7 array Operators 167
7.8 NumPy Calculation Methods 169
7.9 Universal Functions 170
7.10 Indexing and Slicing 171
7.11 Views: Shallow Copies 173
7.12 Deep Copies 174
7.13 Reshaping and Transposing 175
7.14 Intro to Data Science: pandas Series and DataFrames 177
7.15 Wrap-Up 189
Chapter 8: Strings: A Deeper Look 191
8.1 Introduction 192
8.2 Formatting Strings 193
8.3 Concatenating and Repeating Strings 196
8.4 Stripping Whitespace from Strings 197
8.5 Changing Character Case 197
8.6 Comparison Operators for Strings 198
8.7 Searching for Substrings 198
8.8 Replacing Substrings 199
8.9 Splitting and Joining Strings 200
8.10 Characters and Character-Testing Methods 202
8.11 Raw Strings 203
8.12 Introduction to Regular Expressions 203
8.13 Intro to Data Science: Pandas, Regular Expressions and Data Munging 210
8.14 Wrap-Up 214
Chapter 9: Files and Exceptions 217
9.1 Introduction 218
9.2 Files 219
9.3 Text-File Processing 219
9.4 Updating Text Files 222
9.5 Serialization with JSON 223
9.6 Focus on Security: pickle Serialization and Deserialization 226
9.7 Additional Notes Regarding Files 226
9.8 Handling Exceptions 227
9.9 finally Clause 231
9.10 Explicitly Raising an Exception 233
9.11 (Optional) Stack Unwinding and Tracebacks 233
9.12 Intro to Data Science: Working with CSV Files 235
9.13 Wrap-Up 241
Chapter 10: Object-Oriented Programming 243
10.1 Introduction 244
10.2 Custom Class Account 246
10.3 Controlling Access to Attributes 249
10.4 Properties for Data Access 250
10.5 Simulating “Private” Attributes 256
10.6 Case Study: Card Shuffling and Dealing Simulation 258
10.7 Inheritance: Base Classes and Subclasses 266
10.8 Building an Inheritance Hierarchy; Introducing Polymorphism 267
10.9 Duck Typing and Polymorphism 275
10.10 Operator Overloading 276
10.11 Exception Class Hierarchy and Custom Exceptions 279
10.12 Named Tuples 280
10.13 A Brief Intro to Python 3.7’s New Data Classes 281
10.14 Unit Testing with Docstrings and doctest 287
10.15 Namespaces and Scopes 290
10.16 Intro to Data Science: Time Series and Simple Linear Regression 293
10.17 Wrap-Up 301
Chapter 11: Natural Language Processing (NLP) 303
11.1 Introduction 304
11.2 TextBlob 305
11.3 Visualizing Word Frequencies with Bar Charts and Word Clouds 319
11.4 Readability Assessment with Textatistic 324
11.5 Named Entity Recognition with spaCy 326
11.6 Similarity Detection with spaCy 327
11.7 Other NLP Libraries and Tools 328
11.8 Machine Learning and Deep Learning Natural Language Applications 328
11.9 Natural Language Datasets 329
11.10 Wrap-Up 330
Chapter 12: Data Mining Twitter 331
12.1 Introduction 332
12.2 Overview of the Twitter APIs 334
12.3 Creating a Twitter Account 335
12.4 Getting Twitter Credentials–Creating an App 335
12.5 What’s in a Tweet? 337
12.6 Tweepy 340
12.7 Authenticating with Twitter Via Tweepy 341
12.8 Getting Information About a Twitter Account 342
12.9 Introduction to Tweepy Cursors: Getting an Account’s Followers and Friends 344
12.10 Searching Recent Tweets 347
12.11 Spotting Trends: Twitter Trends API 349
12.12 Cleaning/Preprocessing Tweets for Analysis 353
12.13 Twitter Streaming API 354
12.14 Tweet Sentiment Analysis 359
12.15 Geocoding and Mapping 362
12.16 Ways to Store Tweets 370
12.17 Twitter and Time Series 370
12.18 Wrap-Up 371
Chapter 13: IBM Watson and Cognitive Computing 373
13.1 Introduction: IBM Watson and Cognitive Computing 374
13.2 IBM Cloud Account and Cloud Console 375
13.3 Watson Services 376
13.4 Additional Services and Tools 379
13.5 Watson Developer Cloud Python SDK 381
13.6 Case Study: Traveler’s Companion Translation App 381
13.7 Watson Resources 394
13.8 Wrap-Up 395
Chapter 14: Machine Learning: Classification, Regression and Clustering 397
14.1 Introduction to Machine Learning 398
14.2 Case Study: Classification with k-Nearest Neighbors and the Digits Dataset, Part 1 403
14.3 Case Study: Classification with k-Nearest Neighbors and the Digits Dataset, Part 2 413
14.4 Case Study: Time Series and Simple Linear Regression 420
14.5 Case Study: Multiple Linear Regression with the California Housing Dataset 425
14.6 Case Study: Unsupervised Machine Learning, Part 1–Dimensionality Reduction 438
14.7 Case Study: Unsupervised Machine Learning, Part 2–k-Means Clustering 442
14.8 Wrap-Up 455
Chapter 15: Deep Learning 457
15.1 Introduction 458
15.2 Keras Built-In Datasets 461
15.3 Custom Anaconda Environments 462
15.4 Neural Networks 463
15.5 Tensors 465
15.6 Convolutional Neural Networks for Vision; Multi-Classification with the MNIST Dataset 467
15.7 Visualizing Neural Network Training with TensorBoard 486
15.8 ConvnetJS: Browser-Based Deep-Learning Training and Visualization 489
15.9 Recurrent Neural Networks for Sequences; Sentiment Analysis with the IMDb Dataset 489
15.10 Tuning Deep Learning Models 497
15.11 Convnet Models Pretrained on ImageNet 498
15.12 Wrap-Up 499
Chapter 16: Big Data: Hadoop, Spark, NoSQL and IoT 501
16.1 Introduction 502
16.2 Relational Databases and Structured Query Language (SQL) 506
16.3 NoSQL and NewSQL Big-Data Databases: A Brief Tour 517
16.4 Case Study: A MongoDB JSON Document Database 520
16.5 Hadoop 530
16.6 Spark 541
16.7 Spark Streaming: Counting Twitter Hashtags Using the pyspark-notebook Docker Stack 551
16.8 Internet of Things and Dashboards 560
16.9 Wrap-Up 571
Index 573