The Infoq Podcast

Shubha Nabar is a senior director of data science for Salesforce Einstein. Prior to working for Salesforce, she was a data scientist at LinkedIn and Microsoft. In the podcast she discusses Salesforce Einstein and the problem space that they are trying to solve, explores the differences between enterprise and consumer for machine learning, and then talks about the Optimus Prime Scala library that they use in Salesforce. Why listen to this podcast: * The volume of data, and hardware advances have made it possible to do machine learning to do them a lot faster. * AI is a science of building intelligent software, encompassing many aspects of intelligence that we tend to think of as human. * If you can’t measure something, you can’t fix it. * You have to think about what you can automate, rather than having a human to try and engineer out all those features. * Get feedback on design. Nora Jones, a senior software engineer on Netflix’ Chaos Team, talks with Wesley Reisz about what Chaos Engineering means today.

This week's podcast features Simon Brown well known for his work training software architects. Topics include the differences between a tech lead and an architect, how much documentation is enough and what that looks like in a continuous delivery environment. What you'll learn on this podcast: • As an industry we seem to have lost our knowledge of how to do architecture well in the context of modern agile software teams. • Architecture is about the expensive decisions; things that are costly to change later. • Ideally architects should code in the production code base. If you are not able to do this at least be involved in quality reviews and peer reviews in the production code so you can get feedback on your designs. • It is often said the the code is the only documentation you need but the code can’t tell you everything. You do need to document the things you can’t get from the code such as the architectural drivers, they key quality attributes and so on along with some high level diagrams and how you oper

This week's podcasts features Yao Yue of Twitter. Yao spent the majority of her career working on caching systems at Twitter. She has since created a performance team that deals with edge performance outliers often exposed by the enormous scale of Twitter. In this podcast, she discusses standing up the performance team, thoughts on instrumenting applications, and interesting performance issues (and strategies for solving them) they’ve seen at Twitter. Why listen to this podcast: * Performance problems can be caused by a few machines running slowly causing cascading failure * Aggregating stats on a minute-by-minute basis can be an effective way of monitoring thousands of servers * Being able to record second-by-second is often too expensive to centrally aggregate, but can be stored locally * Distinguishing between request timeout and connection/network timeouts is important to prevent thundering herds * With larger scale organisations, having dedicated performance teams helps centralise skills to solve perfo

On the InfoQ Podcast this week, Wes Reisz talks with the Queen of Patterns, Linda Rising. Linda discusses her thoughts on the importance of patterns, she answers questions about what really is a pattern, and how she became involved in working with them. Throughout the podcast she discusses a variety of organizational and personal patterns and finally wraps with patterns to apply when driving change and innovation. Why listen to this podcast: - You have to realise that there’s nothing you can do about other people. The only person you can affect is yourself. - A pattern is not a band-aid that you use once. You use it in a context where you use it in conjunctions with other patterns. - Take baby steps when driving change in an organisation, and seek out a pocket of receptive people to drive it. - Slack is an important part to have in life, so that if something comes along you can absorb it without having to stop doing something else. - Listen, Listen, Listen. More on this: Quick scan our curated show no

Wesley Reisz talks with Sam Newman about microservices. They explore the current state of the art with regards the architectural style and corresponding tooling and deployment platforms. They then discuss how microservices increase the surface area of where sensitive information can be read or manipulated, but also have the potential to create systems that are more secure. Why listen to this podcast: - Different organisations have different risk appetites for new technology, so what may be appropriate for one organisation may not be appropriate technology choices for another. - If you are deploying micro services then you need to know why you are doing it and what benefits you expect to get from deploying them. - Micro services are defined by their independently deployable units rather than their size. - Using a cryptographic token that is verifiable off line is a common pattern for passing authentication contexts around to different services. - Serverless architectures redeuce the need to monitor s

Wesley Reisz talks with Jessica Kerr about her focus on developer productivity. Topics include her work at Atomist building Slack Chatbots, an approach to categorizing Yak Shaving (in an effort to prioritize and automate development dependencies), how an innovation culture drives diversity, and, finally, the role of 10x developers in the lifecycle of a company or product. Why listen to this podcast: - There are five kinds of Yak to shave - Atomist uses a Slack chatbot to automate and track commits, builds, push requests etc. - Agile retrospectives are a great way to encourage an innovation culture - Diverse teams flourish in innovation cultures - 10x developers are great for launching products, but teams are needed as products scale up More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2uO60PR You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like I

Werner Schuster talks to Martin Hadley, data scientist at University of Oxford. They discuss the state of the R language, the rich R ecosystem that covers development (RStudio), notebooks for publication (R Notebooks, RPubs), writing web apps (Shiny), and the pros/cons of the different data frames implementations. Why listen to this podcast: - R is the tool for working with rectangular data - Modern data frame implementations are Tibble and data.table (for large amounts of data) - RMarkdown and R Notebooks allow to explore data and then publish it the results and (interactive) visualization - Use Shinyapps to publish server side R applications - Tidyverse is the place to look for modern R packages More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2twOXWJ You can also subscribe to the InfoQ newsletter to receive weekly updates on the hotest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow

In this podcast Charles Humble talks to Sylvan Clebsch, who is the designer of the actor-model language Pony programming and now works at Microsoft Research in Cambridge in the Programming Language Principles group. They talk about the inspirations behind Pony, how the garbage collector avoids stop-the-world pauses, the queuing systems, work scheduler, and formal verification. Why listen to this podcast: * Pony scales from a Raspberry Pi through a 64 core half terabyte machine to a 4096 core SGI beast * An actor has a 256-byte overhead, so creating hundreds of thousands of actors is possible * Actors have unbounded queues to prevent deadlock * Each actor garbage collects its own heap, so global stop-the-world pauses are not needed * Because the type system is data-race free, it’s impossible to have concurrency problems in Pony More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2tZXcKE You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from

Why listen to this podcast: - Kotlin is an officially supported language on Google Android platforms - Kotlin Native and Kotlin JS will allow code reuse between server, client and mobile devices - Type safety means that references can be checked for nullability Great tooling is a driver in what kind of language features are (and aren’t) adopted - Coroutines provide a way of creating maintainable asynchronous systems More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2sHyxqQ You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitter.com/InfoQ Follow on LinkedIn: www.linkedin.com/company/infoq Want to see extented shownotes? Check the landing page on InfoQ: http://bit.ly/2sHyxqQ

Wesley Reisz talks to Sid Anand, a data architect at cybersecurity company Agari, about building cloud-native data pipelines. The focus of their discussion is around a solution Agari uses that is built from Amazon Kinesis Streams, serverless functions, and auto scaling groups. Sid Anand is an architect at Agari, and a former technical architect at eBay, Netflix, and LinkedIn. He has 15 years of data infrastructure experience at scale, is a PMC for Apache Airflow, and is also a program committee chair for QCon San Francisco and QCon London. Why listen to this podcast - Real-time data pipeline processing is very latency sensitive - Micro-batching allows much smaller amounts of data to be processed - Use the appropriate data store (or stores) to support the use of the dataIngesting data quickly into a clean database with minimal indexes can be fast - Communicate using a messaging system that supports schema evolution More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2rJU9nB You can

Wesley Reisz talks to Sachin Kulkarni, Director of Engineering at Facebook, about the engineering challenges for Facebook live, and how it compares to the video upload platform at Facebook. Why listen to this podcast: - Facebook Infrastructure powers the board family of apps including the Facebook app, Messenger and Instagram. It is largely a C++ shop. There is some Java and Python, and the business logic is all done in PHP. The iOS apps are written in Objective C and the Android apps are in Java. - The video infra team at Facebook builds the video infrastructure across the whole company. Projects include a distributed video encoding platform which results in low latency video encoding, video upload and ingest. - Facebook Live does encoding on both the client and the server. The trade-off between encoding on the client side and the server side is mostly around the quality of the video vs. latency and reliability. - Facebook gets around 10x speed-up by encoding data in parallel compared to serial

Wesley Reisz talks to Martijn Verburg, co-founder of the London Java Community and CEO of jClarity, about the JCP EC “no” vote on the Java Platform Module System (JPMS), which is due to be shipped as part of Java 9. The talk about what JPMS offers, how it works, what the no vote means and what happens next. Why listen to this podcast: - Jigsaw isn’t dead - The “no” vote was based on the submission being a bit early, and without expert group consensus that it should be submitted - Since the vote started, several amendments have been made which addressed some of the concerns listed by those who voted “no” - Daily calls with the expert group and interested parties will work to resolve the outstanding issues promptly - A resubmission is due within 30 days with a future vote expected to go through More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2q20esc You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development.

Wesley Reisz talks to Daniel Bryant on moving from monoliths to micro-services, covering bounded contexts, when to break up micro-services, event storming, practices like observability and tracing, and more. Why listen to this podcast: - Migrating a monolith to micro-services is best done by breaking off a valuable but not critical part first. - Designing a greenfield application as micro-services requires a strong understanding of the domain. - When a request enters the system, it needs to be tagged with a correlation id that flows down to all fan-out service requests. - Observability and metrics are essential parts to include when moving micro-services to production. - A service mesh allows you to scale services and permit binary transports without losing observability. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2pFYBiT You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Sub

Rossen Stoyanchev talks to Wesley Reisz about blocking and non-blocking architectures, upcoming changes in Spring including Spring WebFlux, the reactive web stack in Spring framework 5, due this summer. He also discusses the differences between rxJava and Reactor. Why listen to this podcast: - Spring Framework 5 is due to be released June 25 2017 - Spring Web Flux provides a web programming model designed for asynchronous APIs - Back-pressure is important in a server environment; less so within a UI environment - It’s possible to use a Spring Web Flux client within a Spring MVC applciation - Managing sets of thread pools is more complicated than having a scalable asynchronous system More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2pPgq0G You can subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq Subscribe: www.youtube.com/infoq Like InfoQ on Facebook: bit.ly/2jmlyG8 Follow on Twitter: twitte

Why listen to this podcast: - Using a compiler to catch errors at compile time instead of at runtime means much easier refactoring of code. - Incrementally replacing small parts of an existing JavaScript application with Elm is a safer strategy than trying to write an entirely new application in Elm - Elm packages are semantically versioned and gated by the publishing process, so minor versions cannot remove functionality without bumping the major version. - The UI in an Elm application results in messages that transform the immutable state of the application; this allows a debugger to view the state transitions and the messages that triggered them, including record and replay of those messages. - Elm has been benchmarked as being faster than Angular and React whilst being smaller code, which is attributed to the immutable state and pure functional elements. More on this: Quick scan our curated show notes on InfoQ http://bit.ly/2qmS2CT You can also subscribe to the InfoQ newsletter to receive weekly u

Jean Barmash is Director of Engineering at Compass, Founder & Co-Organizer, NYC CTO School Meetup. Live in New York City. He has over 15 years of experience in software industry, and has been part of 4 startups over the last seven years, 3 as CTO / VPE and one of which he co-founded. Prior to his entrepreneurial adventures, Jean held a variety of progressively senior roles in development, integration consulting, training, and team leadership. He worked for such companies as Trilogy, Symantec, Infusion and Alfresco, consulting to Fortune 100 companies like Ford, Toyota, Microsoft, Adobe, IHG, Citi, BofA, NBC, and Booz Allen Hamilton. Jean will speak at QCon New York 2017: http://bit.ly/2nN7KKo Why listen to this podcast: - The Compass backend is mostly written in Java and Python, with Go increasingly a first class language. The main reason for Go being added was developer productivity. - The app is based on a Microservices architecture with around 40-50 services in total. - Binary RPC, originally Thrift

Eric Horesnyi, CEO @streamdata.io, talks to Charles Humble about how hedge funds are applying deep learning as an alternative to the raw speed favoured by HFT to try and curve the market. Why listen to this podcast: - Streamdata.io was originally built for banks and brokers, but more recently hedge funds have begun using the service. - Whilst Hedge Funds like Renaissance Technologies have been using mathematical approaches for some time deep learning is now being applied to markets. Common techniques such as gradient descent and back propagation apply equally well to market analysis. - The data sources used are very broad. As well as market data the network might be using, sentiment analysis from social networks, social trading data, as well as more unusual data such as retail data, and IoT sensors from farms and factories. - By way of contrast High Frequency Trading focusses on latency. From an infrastructure stand-point you can play with propagation time, Serilization (the thickness of the pipe), an

Greg Murphy is the COO of Gamesparks, a cloud-based platform providing and a rich mobile back-end service for game developers to engage with their users. Greg takes us inside Gamesparks discussing the architecture, machine learning and what it’s like to launch in the China market. Why listen to this podcast: Gamesparks Engagement Engine Tuning the Gaming Experience The Architecture SDK’s and Real-time Data Transfers Server-side Scripts Managing Noisy neighbours and Security The Developer Experience Deploying Across Three Cloud Providers Machine Learning The China Market Notes and links can be found on: http://bit.ly/2neCjEV You can also subscribe to the InfoQ newsletter to receive weekly updates on the hottest topics from professional software development. bit.ly/24x3IVq

Wesley Reisz talks to Slava Oks, who has worked at Microsoft for over 20 years on flagship products, including SQL Server. He also led the kernel team who worked on the Midori operating system. More recently, he has worked on bringing SQL Server to Linux. Why listen to this podcast: - Microsoft SQL Server runs on Linux through a containerised approach called Drawbridge - Drawbridge implements a Linux loader and a minimal set of ABI calls to allow an in-process NT user mode kernel to run - SQL Server runs on top of a SQL platform layer (called SQL OS) that could be ported to run on Drawbridge - SQL Server had supportability commands added to allow the state of the system to be measured with SQL calls - A number of efficiency gains were applied to both the Drawbridge components and the SQL Server code to bring performance to within 20% of the equivalent process running on Windows Notes and links can be found on: http://bit.ly/2kVfatX Drawbridge ABIs Security LibOS Linux SQL OS Supportability Some assem

Jonas Bonér, CTO of LightBend and creator Akka, discusses using Akka when developing distributed systems. He talks about the Actor Model, and how every Microservice needs to be viewed as a system to be successful. Why listen to this podcast: - Akka is JVM-based framework design for developing distributed systems leveraging the Actor Model - an approach for writing concurrent systems that treat actors as universal primitives and the most successful model with abstraction has been streaming - Circuit breakers in Akka are a backup and retry policy; they protect you by capturing failure data and allow you to roll back - Every Microservice needs to be viewed as a system, it needs to have multiple parts that run on different machines in order to function and be fully resilient - thus is a Microsystem - Two different trends have emerged when it comes to hardware and environments: one is the trend toward Multi-core, the is a movement toward virtualized environments and the cloud - Saga pattern of managing lo