Abstract

We will briefly cover the Kroger Company and its digital properties, along with its current recommendation systems and need for a new one. We will then move into a deep dive of the system we developed, covering the Python APIs for large-scale data processing tool Spark, and the underlying Hadoop Distributed File System (HDFS) - focusing on how we utilized each in our implementation. We’ll also discuss process scheduling and coordination via Apache Airflow, along with its Python API and use of Python eggs. Finally, we will show the recommendation system in action, and discuss plans for testing and improvement.

Talk will be organized as follows:

Intro - Context Setting (5 min)

• What is Kroger?

• What is 84.51?

  • What is Digital Personalization at 84.51?

• Landscape: Kroger’s digital properties

• Typically use Retention-focused Recommendation Systems
  • These tend to work extremely well with grocery’s cyclic purchase cycles
• Need for Acquisition-based Recommendation System

Body - Technical Deep Dive (20 min)

New Product Recommender - Ensemble Recommendation System

Part 0:

• Hadoop & Spark, and their Python API

Part 1: Collaborative Filtering

• Overview
• Training (PySpark)
• Implementation (PySpark)
  • Roadblocks

Part 2: Regularized Regression

• Overview
• Training
• Implementation (PySpark)

Part 3: Process Scheduling

• Overview of Airflow
  • Directed Acyclic Graphs
  • Python directive script layout
  • Python Eggs

Part 4: Live view of system on kroger.com

Conclusion (5 min)

• Next Steps - Testing