HERE'S HOW A LEADING IOT COMPANY REINED IN HIGH-VOLUME DATA SOURCES AND MANAGED LOGS BETTER AT SCALE.

Oracle Cloud | Log Management | RBAC | SSO | Alerts

Company details

The client is an established fintech company that invests in and partners with financial institutions around the world with the aim of fostering financial inclusion for unserved and underserved communities. They facilitate management consulting and technology to support their partners and new business initiatives.

Industry/vertical: Smart Home & IoT Solutions

The Challenges

Owing to their increasing popularity and demand, our client was acquiring new customers at a rapid pace. But as their customer base grew, so did their requirements for better data management at scale, longer data retention, and affordable storage costs. Their systems consistently generated 3 TB of log data per day, which was impossible to ingest using a SaaS solution without significantly increasing their IT spend. They also wanted to own their data to achieve better security, governance, and compliance and ideally wanted a SaaS-like experience with a PaaS solution.

The client evaluated Elasticsearch but soon discovered that the raw cost of disk storage that Elasticsearch consumes would quickly exceed their current SaaS spend. They also realized that they’d need to build significant expertise in Elasticsearch to run the solution in-house. Elasticsearch also posed a challenge to have flexibility with data retention on-demand without backing a costly storage project. The client then switched to a SaaS solution named LogEntries. However, LogEntries could not scale beyond 400 GB/day due to the high storage costs involved.

The client was also looking to open their log data to a key OEM partner while restricting access to logs for the OEM. They need RBAC capabilities embedded in their log management system to help them control and govern log access within a single platform. With their current log management stack not meeting their data management needs nor being financially reasonable, they were looking for a single platform that would do it all.

The Need

Log data aggregation, storage, and
management at scale
Log aggregation and unification from multiple data sources
Enhanced data ingestion rates with the ability to manage surges and spikes
Reasonable log storage costs and longer retention
Enterprise-grade RBAC to control data access across internal and partner teams

The LOGIQ Advantage

Our client soon discovered LOGIQ and found it was the exact type of data management platform they were looking for. When asked to choose between LOGIQ SaaS and PaaS, our client decided to go down the PaaS route so that they’d be able to exercise closer management and control over their LOGIQ instance. With LOGIQ PaaS, our clients could also exercise better compliance by keeping all data and associated systems on-prem.

LOGIQ provided flexibility with data retention on-demand with an easy 1-click experience using cloud provider lifecycle policy management on their S3 compatible bucket. LOGIQ’s built-in RBAC control at a namespace level allowed developer and OEM teams to co-exist on the same platform while accessing the data they needed. LOGIQ provided a separate cluster for managing OEM data while allowing multi-cluster management of customer clusters via a single user interface.

How LOGIQ Helped

We deployed LOGIQ PaaS within their existing Oracle Cloud ecosystem. A dedicated support team from LOGIQ manages the infrastructure needed for LOGIQ PaaS within the client’s existing Oracle Cloud accounts. We first set up SSO by leveraging the client’s Okta setup. Our native integrations with Prometheus and AWS Athena helped quickly set up log ingestion from these data sources.

We also set up alerts within LOGIQ and integrated them with the client’s Opsgenie ITOM so that alerts generated within LOGIQ are forwarded instantly to Opsgenie for further action and automation.

Results

Our client instantly witnessed tremendous performance gains in comparison to the previous solutions they used. They went from being limited to ingesting and analyzing 400 GB/day on their old system to consistently consuming 3 TB/day with LOGIQ without any limitations. Since LOGIQ enables the use of S3 as the primary storage layer, they could break away from the 7-day limit for data retention and retain data for as long as they liked at 1/4th the cost. Our integrations helped reel in disparate data sources and converge all of their data in one platform. Our integration with Opsgenie helped add dimensionality to the issues and failures they usually were alerted for. LOGIQ’s RBAC capabilities also helped our client ensure that the right teams had the right level of access to log data and helped internal and partner teams analyze the data they needed to identify and debug issues and threats

Key Statistics

4X more data unified per month
3 Billion logs ingested per day
Peak load of 160 GB/h
100% uptime over 90 days
50% TCO reduction
8x ROI improvement
#ZeroStorageTax with 1-click
extension of retention duration