Published by Addison-Wesley Professional (January 13, 2022) © 2022
David Farley
Improve Your Creativity, Effectiveness, and Ultimately, Your Code
In Modern Software Engineering, continuous delivery pioneer David Farley helps software professionals think about their work more effectively, manage it more successfully, and genuinely improve the quality of their applications, their lives, and the lives of their colleagues.
Writing for programmers, managers, and technical leads at all levels of experience, Farley illuminates durable principles at the heart of effective software development. He distills the discipline into two core exercises: learning and exploration and managing complexity. For each, he defines principles that can help you improve everything from your mindset to the quality of your code, and describes approaches proven to promote success.
Farley's ideas and techniques cohere into a unified, scientific, and foundational approach to solving practical software development problems within realistic economic constraints. This general, durable, and pervasive approach to software engineering can help you solve problems you haven't encountered yet, using today's technologies and tomorrow's. It offers you deeper insight into what you do every day, helping you create better software, faster, with more pleasure and personal fulfillment.
In Modern Software Engineering, continuous delivery pioneer David Farley helps software professionals think about their work more effectively, manage it more successfully, and genuinely improve the quality of their applications, their lives, and the lives of their colleagues.
Writing for programmers, managers, and technical leads at all levels of experience, Farley illuminates durable principles at the heart of effective software development. He distills the discipline into two core exercises: learning and exploration and managing complexity. For each, he defines principles that can help you improve everything from your mindset to the quality of your code, and describes approaches proven to promote success.
Farley's ideas and techniques cohere into a unified, scientific, and foundational approach to solving practical software development problems within realistic economic constraints. This general, durable, and pervasive approach to software engineering can help you solve problems you haven't encountered yet, using today's technologies and tomorrow's. It offers you deeper insight into what you do every day, helping you create better software, faster, with more pleasure and personal fulfillment.
- Clarify what you're trying to accomplish
- Choose your tools based on sensible criteria
- Organize work and systems to facilitate continuing incremental progress
- Evaluate your progress toward thriving systems, not just more "legacy code"
- Gain more value from experimentation and empiricism
- Stay in control as systems grow more complex
- Achieve rigor without too much rigidity
- Learn from history and experience
- Distinguish "good" new software development ideas from "bad" ones
Foreword xvii
Preface xxi
Acknowledgments xxv
About the Author xxvii
Part I: What Is Software Engineering? 1
Chapter 1: Introduction 3
Engineering--The Practical Application of Science 3
What Is Software Engineering? 4
Reclaiming "Software Engineering" 5
The Birth of Software Engineering 7
Shifting the Paradigm 8
Summary 9
Chapter 2: What Is Engineering? 11
Production Is Not Our Problem 11
Design Engineering, Not Production Engineering 12
A Working Definition of Engineering 17
Engineering != Code 17
Why Does Engineering Matter? 19
The Limits of "Craft" 19
Precision and Scalability 20
Managing Complexity 21
Repeatability and Accuracy of Measurement 22
Engineering, Creativity, and Craft 24
Why What We Do Is Not Software Engineering 25
Trade-Offs 26
The Illusion of Progress 26
The Journey from Craft to Engineering 27
Craft Is Not Enough 28
Time for a Rethink? 28
Summary 30
Chapter 3: Fundamentals of an Engineering Approach 31
An Industry of Change? 31
The Importance of Measurement 32
Applying Stability and Throughput 34
The Foundations of a Software Engineering Discipline 36
Experts at Learning 36
Experts at Managing Complexity 37
Summary 38
Part II: Optimize for Learning 41
Chapter 4: Working Iteratively 43
Practical Advantages of Working Iteratively 45
Iteration as a Defensive Design Strategy 46
The Lure of the Plan 48
Practicalities of Working Iteratively 54
Summary 55
Chapter 5: Feedback 57
A Practical Example of the Importance of Feedback 58
Feedback in Coding 60
Feedback in Integration 61
Feedback in Design 63
Feedback in Architecture 65
Prefer Early Feedback 67
Feedback in Product Design 68
Feedback in Organization and Culture 68
Summary 70
Chapter 6: Incrementalism 71
Importance of Modularity 72
Organizational Incrementalism 73
Tools of Incrementalism 74
Limiting the Impact of Change 76
Incremental Design 77
Summary 79
Chapter 7: Empiricism 81
Grounded in Reality 82
Separating Empirical from Experimental 82
"I Know That Bug!" 82
Avoiding Self-Deception 84
Inventing a Reality to Suit Our Argument 85
Guided by Reality 88
Summary 89
Chapter 8: Being Experimental 91
What Does "Being Experimental" Mean? 92
Feedback 93
Hypothesis 94
Measurement 95
Controlling the Variables 96
Automated Testing as Experiments 97
Putting the Experimental Results of Testing into Context 98
Scope of an Experiment 100
Summary 100
Part III: Optimize for Managing Complexity 103
Chapter 9: Modularity 105
Hallmarks of Modularity 106
Undervaluing the Importance of Good Design 107
The Importance of Testability 108
Designing for Testability Improves Modularity 109
Services and Modularity 115
Deployability and Modularity 116
Modularity at Different Scales 118
Modularity in Human Systems 118
Summary 120
Chapter 10: Cohesion 121
Modularity and Cohesion: Fundamentals of Design 121
A Basic Reduction in Cohesion 122
Context Matters 125
High-Performance Software 128
Link to Coupling 129
Driving High Cohesion with TDD 129
How to Achieve Cohesive Software 130
Costs of Poor Cohesion 132
Cohesion in Human Systems 133
Summary 133
Chapter 11: Separation of Concerns 135
Dependency Injection 139
Separating Essential and Accidental Complexity 139
Importance of DDD 142
Testability 144
Ports & Adapters 145
When to Adopt Ports & Adapters 147
What Is an API? 148
Using TDD to Drive Separation of Concerns 149
Summary 150
Chapter 12: Information Hiding and Abstraction 151
Abstraction or Information Hiding 151
What Causes "Big Balls of Mud"? 152
Organizational and Cultural Problems 152
Technical Problems and Problems of Design 154
Fear of Over-Engineering 157
Improving Abstraction Through Testing 159
Power of Abstraction 160
Leaky Abstractions 162
Picking Appropriate Abstractions 163
Abstractions from the Problem Domain 165
Abstract Accidental Complexity 166
Isolate Third-Party Systems and Code 168
Always Prefer to Hide Information 169
Summary 170
Chapter 13: Managing Coupling 171
Cost of Coupling 171
Scaling Up 172
Microservices 173
Decoupling May Mean More Code 175
Loose Coupling Isn't the Only Kind That Matters 176
Prefer Loose Coupling 177
How Does This Differ from Separation of Concerns? 178
DRY Is Too Simplistic 179
Async as a Tool for Loose Coupling 180
Designing for Loose Coupling 182
Loose Coupling in Human Systems 182
Summary 184
Part IV: Tools to Support Engineering in Software 185
Chapter 14: The Tools of an Engineering Discipline 187
What Is Software Development? 188
Testability as a Tool 189
Measurement Points 192
Problems with Achieving Testability 193
How to Improve Testability 196
Deployability 197
Speed 199
Controlling the Variables 200
Continuous Delivery 201
General Tools to Support Engineering 202
Summary 203
Chapter 15: The Modern Software Engineer 205
Engineering as a Human Process 207
Digitally Disruptive Organizations 207
Outcomes vs. Mechanisms 210
Durable and Generally Applicable 211
Foundations of an Engineering Discipline 214
Summary 215
Index 217
Preface xxi
Acknowledgments xxv
About the Author xxvii
Part I: What Is Software Engineering? 1
Chapter 1: Introduction 3
Engineering--The Practical Application of Science 3
What Is Software Engineering? 4
Reclaiming "Software Engineering" 5
The Birth of Software Engineering 7
Shifting the Paradigm 8
Summary 9
Chapter 2: What Is Engineering? 11
Production Is Not Our Problem 11
Design Engineering, Not Production Engineering 12
A Working Definition of Engineering 17
Engineering != Code 17
Why Does Engineering Matter? 19
The Limits of "Craft" 19
Precision and Scalability 20
Managing Complexity 21
Repeatability and Accuracy of Measurement 22
Engineering, Creativity, and Craft 24
Why What We Do Is Not Software Engineering 25
Trade-Offs 26
The Illusion of Progress 26
The Journey from Craft to Engineering 27
Craft Is Not Enough 28
Time for a Rethink? 28
Summary 30
Chapter 3: Fundamentals of an Engineering Approach 31
An Industry of Change? 31
The Importance of Measurement 32
Applying Stability and Throughput 34
The Foundations of a Software Engineering Discipline 36
Experts at Learning 36
Experts at Managing Complexity 37
Summary 38
Part II: Optimize for Learning 41
Chapter 4: Working Iteratively 43
Practical Advantages of Working Iteratively 45
Iteration as a Defensive Design Strategy 46
The Lure of the Plan 48
Practicalities of Working Iteratively 54
Summary 55
Chapter 5: Feedback 57
A Practical Example of the Importance of Feedback 58
Feedback in Coding 60
Feedback in Integration 61
Feedback in Design 63
Feedback in Architecture 65
Prefer Early Feedback 67
Feedback in Product Design 68
Feedback in Organization and Culture 68
Summary 70
Chapter 6: Incrementalism 71
Importance of Modularity 72
Organizational Incrementalism 73
Tools of Incrementalism 74
Limiting the Impact of Change 76
Incremental Design 77
Summary 79
Chapter 7: Empiricism 81
Grounded in Reality 82
Separating Empirical from Experimental 82
"I Know That Bug!" 82
Avoiding Self-Deception 84
Inventing a Reality to Suit Our Argument 85
Guided by Reality 88
Summary 89
Chapter 8: Being Experimental 91
What Does "Being Experimental" Mean? 92
Feedback 93
Hypothesis 94
Measurement 95
Controlling the Variables 96
Automated Testing as Experiments 97
Putting the Experimental Results of Testing into Context 98
Scope of an Experiment 100
Summary 100
Part III: Optimize for Managing Complexity 103
Chapter 9: Modularity 105
Hallmarks of Modularity 106
Undervaluing the Importance of Good Design 107
The Importance of Testability 108
Designing for Testability Improves Modularity 109
Services and Modularity 115
Deployability and Modularity 116
Modularity at Different Scales 118
Modularity in Human Systems 118
Summary 120
Chapter 10: Cohesion 121
Modularity and Cohesion: Fundamentals of Design 121
A Basic Reduction in Cohesion 122
Context Matters 125
High-Performance Software 128
Link to Coupling 129
Driving High Cohesion with TDD 129
How to Achieve Cohesive Software 130
Costs of Poor Cohesion 132
Cohesion in Human Systems 133
Summary 133
Chapter 11: Separation of Concerns 135
Dependency Injection 139
Separating Essential and Accidental Complexity 139
Importance of DDD 142
Testability 144
Ports & Adapters 145
When to Adopt Ports & Adapters 147
What Is an API? 148
Using TDD to Drive Separation of Concerns 149
Summary 150
Chapter 12: Information Hiding and Abstraction 151
Abstraction or Information Hiding 151
What Causes "Big Balls of Mud"? 152
Organizational and Cultural Problems 152
Technical Problems and Problems of Design 154
Fear of Over-Engineering 157
Improving Abstraction Through Testing 159
Power of Abstraction 160
Leaky Abstractions 162
Picking Appropriate Abstractions 163
Abstractions from the Problem Domain 165
Abstract Accidental Complexity 166
Isolate Third-Party Systems and Code 168
Always Prefer to Hide Information 169
Summary 170
Chapter 13: Managing Coupling 171
Cost of Coupling 171
Scaling Up 172
Microservices 173
Decoupling May Mean More Code 175
Loose Coupling Isn't the Only Kind That Matters 176
Prefer Loose Coupling 177
How Does This Differ from Separation of Concerns? 178
DRY Is Too Simplistic 179
Async as a Tool for Loose Coupling 180
Designing for Loose Coupling 182
Loose Coupling in Human Systems 182
Summary 184
Part IV: Tools to Support Engineering in Software 185
Chapter 14: The Tools of an Engineering Discipline 187
What Is Software Development? 188
Testability as a Tool 189
Measurement Points 192
Problems with Achieving Testability 193
How to Improve Testability 196
Deployability 197
Speed 199
Controlling the Variables 200
Continuous Delivery 201
General Tools to Support Engineering 202
Summary 203
Chapter 15: The Modern Software Engineer 205
Engineering as a Human Process 207
Digitally Disruptive Organizations 207
Outcomes vs. Mechanisms 210
Durable and Generally Applicable 211
Foundations of an Engineering Discipline 214
Summary 215
Index 217