New device classes are becoming network-enabled en masse:
- Industrial and factory machines.
- Home appliances.
- Home security and automation.
- Location-based systems, including drones.
Some of these devices can generate data at a staggering rate; a single sensor may produce data volume on the order of an entire enterprise data warehouse… in a single day. Mobile and networked devices have not only become a significant driver of modern Big Data technologies, but are driving evermore innovative and scalable approaches to data management and analytics. Techniques such as stream processing and edge-based filtering and analytics become even more relevant, and tough decisions about ingest and retention policies need to be made.
A holistic solution includes other concerns, as well:
How and when does data get to and from edge devices to server infrastructure? How are issues such as bandwidth, intermittent network connections, and protocol mediation handled? How are devices and data secured?
Edge devices may have limited storage and processing power. Code footprint and efficiency, power, and environmental considerations create a challenging set of design constraints.
What devices are active? Where are they? Which need service? Are there systemic outage patterns that indicate a broader network or platform issue?