The Infoq Podcast

Arun Gupta discusses with Wes Reisz some of the container-focused services that AWS offers, including differentiating ECS and EKS. Arun goes into some detail the role that Amazon Fargate plays and goals behinds EKS. Arun wraps ups discussing some of the open source work that AWS has recently been doing in the container space. Why liste to this podcast: - ECS & EKS are both managed control planes; Amazon Fargate is a technology used to provision clusters. - ECR is the Amazon Container registry (similar to the Docker Registry). - EKS is an opinionated why of running a Kubernetes cluster on AWS. It is a highly available managed control plane available on US East 1 and US West 2 - EKS uses a split account. The control plane runs in an Amazon account and the workers run in customer’s account. - Upstream compatibility is a core tenant of EKS. You can subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.

In this podcast Wes Reisz is talking to Anastasiia Voitova, known as @vixentael in the security communities. She started her career as a mobile application developer, and in recent years has moved to focus mainly on designing and developing graphics software. We’re going to talk about cryptography, how to design libraries to be usable by developers, and designing cryptographic libraries. We’ll also discuss about her talk from the recent QCon New York , called “Making Security Usable”. Why listen to this podcast: - Choosing a good encryption algorithm isn’t enough - the parameters need to be chosen carefully as well - Algorithms like MD5 should not be used for hashing any more - Security is not just the encryption layer - it is the design of the whole system - Backups should be encrypted as well - Logs may contain sensitive GDPR data and need to be processed accordingly More on this: Quick scan our curated show notes on InfoQ https://bit.ly/2yRWdQc You can also subscribe to the InfoQ newsletter to rec

On today’s podcast, Wes Reisz talks to Matt Klein about Envoy. Envoy is a modern, high performance, small footprint edge and service proxy. While it was originally developed at Lyft (and still drives much of their architecture), it is a fully open source driven project. Matt addresses on this podcast what he sees as the major design goals of Envoy, answers questions about a sidecar performance impact, discusses observability, and thinks out loud on the future of Envoy. Why listen to this podcast: - Envoy’s goal is to abstract the network from application programmers. It’s really about helping application developers focus on building business logic and not on the application plumbing. - Envoy is a large community driven project, not a cohesive product that does one thing. It can be used as a foundational building blocks to extend into a variety of use cases, including as an edge proxy, as a service mesh sidecar, and as a substrate for building new products. - While there is performance cost for using sidecar

On this podcast, Pam Selle (an engineer for IOPipe who builds tooling for serverless observability) talks about the case for serverless and the challenges for developing observability solutions. Some of the things discussed on the podcast include tips for creating boundaries between serverless and non-serverless resources and how to think of distributed tracing in serverless environments. Why listen to this podcast: - Coca Cola was able to see a productivity gain of 29% by adopting serverless (as measured by the amount of time spent on business productivity applications). - Tooling for serverless is often a challenge because resources are ephemeral. To address the ephemeral nature of serverless, you need to think about what information you will need to log ahead of time. - Monitoring should focus on events important to the business. - Build barriers between serverless and flat scaling non-serverless resources to prevent issues. Queues are an example of ways to protect flat scaling resources. - In-memory cac

The Serverless Framework is quickly becoming one of the more popular frameworks used in managing serverless deployments. David Wells, an engineer working on the framework, talks with Wes Reisz about serverless adoption and the use of the open source Serverless Framework. On this week’s podcast, the two dive into what it looks like to use the tool, the development experience, why a developer might want to consider a tool like the serverless framework, and finally wraps up with what the tool offers in areas like CI/CD, canaries, and blue/green deployment. Why listen to this podcast: - Serverless allows you to focus on the core business functionality and less on the infrastructure required to run your systems. - Serverless Framework allows you to simplify the amount of configuration you need for each cloud provider (for example, you can automate much of the configuration required for CloudFormation with AWS) - Serverless Framework is an open source CLI tool that supports all major cloud providers and sev

In this podcast Wes Reisz talks to Colin Eberhardt, the Technology Director at Scott Logic, talks about what WebAssembly (WASM) is, a bit of the history of JavaScript, information about WebAssembly, and plans for WebAssembly 2.0 including the threading model and GC. Why listen to this podcast: - WebAssembly brings another kind of virtual machine to the browser that is a much more low-level language. - One of the goals of WebAssembly is to make a new assembly language that is a compilation target for a wide range of other languages such as C++, Java, C# and Rust. C++ is highly mature, Rust is maturing rapidly. Java and C# are a little further behind because of the lack of garbage collection support in WebAssembly. At some point in the future WebAssemblywill have it’s own garbage collection perhaps by using the Javascript garbage collector. - At runtime you use JavaScript to invoke functions that are exported by your WebAssembly instance. It should be noted that at the moment there is quite a lot of comple

Martin Thompson discusses consensus in distributed systems, and how Aeron uses Raft for clustering in the upcoming release. Martin is a Java Champion with over 2 decades of experience building complex and high-performance computing systems. He is most recently known for his work on Aeron and Simple Binary Encoding (SBE). Previously at LMAX he was the co-founder and CTO when he created the Disruptor.  * Aeron is a messaging system designed for modern multi-core hardware. It is highly performant with a first class design goal of making it easy to monitor and understand at runtime. The product is able to simultaneously achieve the lowest latency and highest throughput of any messaging system available today. Why listen to this podcast: * Aeron uses a binary format on the wire rather than a text based protocol. This is largely done for performance reasons. Text is commonly used in messaging to make debugging simpler but the debugging problem can be solved using tools like Wireshark and the dissectors that

In this podcast, recorded live at QCon.ai, Principal Technical Advisor & QCon Chair Wes Reisz and InfoQ Editor-in-chief Charles Humble chair a panel discussion with Stephanie Yee, data scientist at StitchFix, Matei Zaharia, professor of computer science at Stanford and chief scientist at Data Bricks, Sid Anand, chief data engineer at PayPal, and Soups Ranjan, director of data science at CoinBase. Why listen to this podcast: - Before you start putting a data science team together make sure you have a business goal or question that you want to answer;  If you have a specific question, like increasing lift on a metric, or understanding  customer usage patterns, you know where you can get the data from, and you can then figure out how to organise that data. - You need to make sure you have the right culture for the team - and find people who are excited about solving the business problems and be interested in it.  Also look at the environment you are going to provide.  - Your first hire shouldn’t be a data s

On this week’s podcast, Danny Yuan, Uber’s Real-time Streaming/Forecasting Lead, lays out a thorough recipe book for building a real-time streaming platform with a major focus on forecasting. In this podcast, Danny discusses everything from the scale Uber operates at to what the major steps for training/deploy models in an iterative (almost Darwinistic) fashion and wraps with his advice for software engineers who want to begin applying machine learning into their day-to-day job. Why listen to this podcast: * Uber processes 850,000 - 1.3 million messages per second in their streaming platform with about 12 TB of growth per day. The system’s queries scan 100 million to 4 billion documents per second. * Uber’s frontend is mobile. The frontend talks to an API layer. All services generate events that are shuffled into Kafka. The real-time forecasting pipeline taps into Kafka to processes events and stores the data into Elasticsearch. * There is a federated query layer in front of Elasticsearch to provide OLAP

Sander Mak and Wes Reisz discuss the Java module system and how adoption is going. Topics discussed on this podcast include Java modularity steps / migrations, green field projects, some of the concerns that caused the EC to initially vote no on Java 9, and a new tool for building custom JREs called JLink. Additionally, as Java 10 was recently released a short bit at the end was added to discuss some of the latest news with Java. Why listen to this podcast: • People quickly moved to Java 8 because of features like Streams and Lambdas. Java 9 has a different story around modularity and application architecture. Adoption is slower and more intentional. • Migrating large codebases to use modularity is hard. Many of the projects using modules are greenfield, and those large codebases that are moving now are most often using the classpath. • Jlink is a new command line tool released with Java 9. It allows developers to create their own lightweight, customized JRE for a module-based Java application. • Java versi

Jendrik Joerdening and Anthony Navarro describe how a team of 18 Udacity students entered a self-racing car event   They had very limited experience of building autonomous control systems for vehicles and had just 6 weeks to do it with only 2 days with the physical car.  They describe the architecture, how they co-ordinated a very diverse team, and how they trained the models. Why listen to this podcast: - Last year a team of 18 Udacity Self-Driving Cars students competed at the 2017 Self Racing Cars event held at Thunderhill Raceway in California. - The students had all taken the first term of a three term program on Udacity which covers computer vision and deep learning techniques. - The team was extremely diverse.  They co-ordinated the work via Slack with a team in 9 timezones and 5 different countries.   - The team developed a neural network using Keras and Tensorflow which steered the car based on the input from just one front-facing camera in order to navigate all turns on the racetrack.  - They

On this podcast, we talk with Andrea Magnorsky, who is a tech lead at Goodlord on their engineering squads; she has a background in Scala, C#, and organised conferences. Today we’ll be talking about paradigm shifts. Why listen to this podcast: * A programming paradigm has a loose definition. It’s just about finding a way of doing things. * There are a number of different ways to think about problems - and different paradigms do this in different ways. * To shift paradigms, you have to un-learn some of your instincts. * When adopting a new paradigm if people don’t want to learn anything, then they won’t. * Multiple paradigms help you apply different ways of thinking about solutions to problems because solutions vary across languages. * Quick ways to start gaining knowledge and adoption for new languages are to use a new language as a test harness for your existing code. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2oPFG71 You can also subscribe to the InfoQ newsletter to receive w

On this podcast, Anne Currie joins the tech ethics discussion started on the Theo Schlossnagle podcast from a few weeks ago. Wes Reisz and Anne discuss issues such as the implications (and responsibilities) of the massive amount of scale we have at our fingertips today, potential effects of GDPR (EU privacy legislation), how accessibility is a an example of how we could approach tech ethics in software, and much more. Why listen to this podcast: - Ethics in software today is particularly important because of the scale we have available with cloud native architectures. - Accessibility offers a good approach to how we can evolve the discussion on tech ethics with aspects that include both a carrot and a stick. - Bitcoin mining power consumption is an example of something we never considered to have such negatives. - The key to establishing what we all should and shouldn’t be doing with tech ethics is to start conversations and share our lessons with each other. If you want to find out what every software deve

This week on The InfoQ Podcast Wes Reisz talks with the CTO of Bouyant Oliver Gould. Bouyant is the maker the LinkerD Service Mesh and the recently released Conduit. In the podcast, Oliver defines a service mesh, clarifies the meaning of the data and control plane, discusses what a Service Mesh can offer a Microservice application owners, and, finally, discusses some of the considerations they took into account developing Conduit. Why listen to this podcast: - Service mesh is dedicated infrastructure that handles interservice communication. - There are two components to a service mesh: the data plane handles communication and the control plane is about policy and config. - LinkerD and Conduit are two open service meshes made by Bouyant. Conduit has a small memory footprint and provides a convention over configuration approach to service mesh deployment. - Adopting Rust (language used for implementing the data plane in Conduit) requires thinking of memory differently, and the best way to adopt Rust is to rea

This week's podcast features a chat with Theo Scholossnagle. Theo is the CEO of Circonus and co-chairs the ACM Queue. In this podcast, Theo and Wes Reisz chat about the need for ethical software, and how we as technical leaders should be reasoning about the software we create. Theo says, "it's not about the absence of evil, it's about the presence of good." He challenges us to develop rigor around ethical decisions we make in software just as we do for areas like security. With the incredible implications of machine learning and AI in our future, this week's podcast touches on topics we should all consider in the systems we create. Why listen to this podcast: - The ubiquitous society impact of computers is surfacing the need for deeper conversations on software ethics. - Ethics are a set of constructs and constraints to help us reason about right and wrong. - Algorithmic interpretability of models can be difficult to reason about; however, accountability for algorithms can be enforced in other ways. - Q

On this week’s podcast, Wes Reisz talks with Chris Swan. Chris is the CTO for the global delivery organisation at DXC Technology. Chris is well versed in DevOps, Infrastructure, Culture, and what it means to put all these together. Today’s topics include both DevOps and NoOps, and what Chris calls LessOps, what Operations means in a world of Serverless, where he sees Configuration Management, Provisioning, Monitoring and Logging heading. The podcast then wraps talking about where he sees validating code in a serverless deployment, such as canaries and blue-green deployments. Why listen to this podcast: * Serverless still requires ops - even if the ops aren’t focused on the technology * Even with minimal functions, the amount of configuration may exceed it by a factor of three * Disruptive services often move the decimal point * ML is the ability to make the inferences and AI is the ability to make decisions based on those inferences More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2B

This week’s podcast features a chat with Vitor Olivier. Vitor is a partner at NuBank (a technology-centric bank in Brazil). This podcast hits on topics from several of Nubank’s recent QCon talks and includes things like: Nubank’s stack, functional programming, event sourcing, defining service boundaries, recommendations on reasoning about services, tips (or tweaks) on the second iteration of their initial architecture and more. Why listen to this podcast: - Property-based testing and Schemas (or Clojure.Spec)are complementary. - Clojure’s functional nature and Datomic’s features are a match for Nubank’s requirements. - A (micro)service needs to be able to create the full representation of the core feature it’s handling. - GraphQL is useful to abstract away the distributed system complexity from the mobile (or frontend) developers. - Nubank’s uses a combination of monitoring and sanity checks in real time at various level to keep systems consistent. - Once an invariant is broken, the system will try to f

In this podcast Charles Humble and Wes Reisz talk about Java 9 and beyond, Kotlin, .NET Core 2, the surge in interest in organisational culture, quantum computing and more. Why listen to this podcast: - Java had a big year with Java 9 shipping, Java EE going open-source and moving to Eclipse as EE4J, and IBM open-sprucing J9.  From next year the platform will also be on a bi-annual release cycle with the next two versions (expected to be Java 10 and 11) both shipping during 2018. - Kotlin joined Scala, Clojure, and Groovy as a strong alternative language for the JVM particularly for mobile where it was buoyed by Google’s official blessing of it as a language for Android development at Google IO. - On InfoQ we also saw a big surge in interest around .NET linked to .NET Core 2, and at both InfoQ and at QCon San Fransisco we also saw an upsurge in interest around organizational culture with one of the culture tracks (the Whole Engineer) moving to one of the larger rooms. - We started to see Quantum compute

Gremlin is a Software as a Service that lets you plan, control and undo Chaos engineering experiments built by engineers with experience from Netflix, AWS, Dropbox and others. In this podcast Wes talks to Kolton Andrus about the Gremlin product and architecture and related topics such as running Game Days. You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq

In this podcast, Deam Wampler discusses fast data, streaming, microservices, and the paradox of choice when it comes to the options available today building data pipelines. Why listen to this podcast: * Apache Beam is fast becoming the de-facto standard API for stream processing * Spark is great for batch processing, but Flink is tackling the low-latency streaming processing market * Avoid running blocking REST calls from within a stream processing system - have them asynchronously launched and communicate over Kafka queues * Visibility into telemetry of streaming processing systems is still a new field and under active development * Running the fast data platform is easily launched on an existing or new Mesosphere DC/OS runtime More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2BYTMbI You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Face