Manual Data Entry

Building Datasets

• Data has to be collected.
• Surveys are one way.
• Manual/Automated Data Entry is another.

Manual vs Automated Data Entry

• What is Manual Data Entry?
• What is Automated Data Entry?
• What separates the two?

Why are we talking about them together?

• What do they have in common?
• Must design data organization.
• Data entry follows that model.
  

When do we use it?

• All the time.
• Tracking customer data. (operational/business data) (automated)
  
• Tracking locations of wildlife. (manual)
• Notes on patient care. (manual/automated)

Problems with Manual Data Entry?

• What did the article indicate?
• Slow.
• Error-prone (up to 4%).
• Expensive.
• Human boredom.
• Inconsistent.

Benefits of Manual Data Entry?

• Handles ambiguity well.
• Handles unstructured/poorly structured data well.
• Can be used to train automated systems.
  • Supervised learning.
• Consistency in data entry.

Problems with Automated Data Entry?

• Does not work well will poorly-structured data.
• Prone to false positives and false negatives if not designed well.

Benefits of Automated Data Entry?

• What did the article indicate?
• OCR (optical character recognition) and ICR (Intelligent Character Recognition) are getting better.
• Lowers prices.
• Lowers error rates.

When to use Manual Data Entry?

• When data is unstructured.
• When the information is ambiguous or context specific.
• Gives us better control.

When to use Automated Data Entry?

• When data is well-structured.
• When we can force structure on the data.
• When data follows predictable patterns (for machine learning).

Designing Data Models

• Whatever we choose, we have to organize the data ingestion
• Focus on the smallest unit of data we are interested in.
• Think of these as rows or documents in a database.

Designing Data Models, cont.

• What do we want to track?
• Focus here, then build out.
• We can always aggregate.
• It’s more difficult to break apart.

Example 1: Endangered Birds

• We want to track wild life: endangered birds in a park.
• Manual or Automated?
  • Why?
• What do we need to track?
• We need a list of endangered birds. (know what to look for)
• Nest locations.
  • How do we organize this?
• Bird sightings.
  • How do we organize this?

Example 1: Endangered Birds, cont

• Nest_Locations
  • Bird_ID
  • Location (long/lat)
  • Date
  • Time
  • Observer
  • Notes

• Bird_Sightings
  • Bird_ID
  • Location (long/lat)
  • Date
  • Time
  • Observer
  • Notes

Example 2: Amazon Customer Data

• We want to track customer purchases and views
• Manual or Automated?
  • Why?
• How do we organize this?

Example 2: Amazon Customer Data, cont.

• Customer_Purchases
  • Customer_ID
  • Product_ID
  • Checkout_ID
  • Date
  • Timestamp
  • Price
  • Quantity

• Customer_Views
  • Customer_ID
  • Product_ID
  • Date
  • Timestamp
  • Price