Sky Cone

Week 1: Overview of the ML Lifecycle and Deployment

Section 4: Graded Assessment

1. Deployment

Question 1
You've built a new system for making loan approval decisions. For now, its output is not used in any decision making process, and a human loan officer is solely responsible for deciding what loans to approve. But the system's output is logged for analysis. What is this type of deployment called?

1. Red green deployment
2. Canary deployment
3. Blue green deployment
4. Shadow mode deployment
   

Question 2
On a social media platform, you're rolling out a new anti-spam system to flag and hide spammy posts. Your team decides to roll out the anti-spam filter via a canary deployment, and roll it out to 1% of users initially. Which of these would you advocate for?

1. After a successful canary deployment, begin to implement a shadow mode deployment.
2. Avoid intervening with a rollback until ramp up is complete even if the system isn't working.
3. Monitor that 1% of users' reaction, and if it goes well, flip the switch to send all traffic (100%) to the system.
4. Monitor that 1% of users' reaction, and either gradually ramp up (if it's going well) or rollback (if not)

Question 3
You're building a healthcare screening system, where you input a patient's symptoms, and for the easy cases (such as an obvious case of the common cold) the system will give a recommendation directly, and for the harder cases it will pass the case on to a team of in-house doctors who will form their own diagnosis independently. What degree of automation are you implementing in this example for patient care?

1. Shadow mode
2. Full Automation
3. Human only
4. Partial Automation

Question 4
You have built and deployed an anti-spam system that inputs an email and outputs either 0 or 1 based on whether the email is spam. Which of these will result in either concept drift or data drift?

1. Spammers trying to change the wording used in emails to get around your spam filter.
2. Cloud computational costs going down, resulting in a lower cost to process each email received.
3. Updating a monitoring dashboard to keep track of new metrics.
4. None of these will result in either concept drift or data drift.

Question 5
Which of these statements is a more accurate description of deployment?

1. Because deployment is a high stakes event, it's critical to design the right system, so that immediately after launch it will immediately work reliably and scale effectively.
2. It is an iterative process, where you should expect to make multiple adjustments (such as metrics monitored using dashboards or percentage of traffic served) to work towards optimizing the system.