Watching the Inventives

« Back to Blog

(tip of the cap to Elvis Costello)

Analyst Community Value

• Among this cohort, roughly 10% are high value, 20% are moderate value and the remainder are “learning their craft” or have inherited processes from predecessors and are not actively involved in tweaking or refining these processes.
• Among these analysts, fewer than 10% of their workloads use high-powered analytic techniques, and fewer yet use any machine learning techniques, leaving an enormous gap in terms of business value potential for the enterprise.
• 15% of these analysts are high-value information producers, who build information assets used by at least 2 or more downstream consumers. These producers average roughly 15-30 downstream consumers.
• Some organizations have a higher ratio of analysts who are both producers and consumers, and we have found this is a marker of an organization that has an effective culture for information sharing and cross-pollination.

Workload Portfolio Value

• Of these workloads, only 20% produce high business value to the enterprise.
• These workloads are owned by only 10-15% of the enterprise’s analyst community.
• We define high business value using a workload scorecard assessed across multiple dimensions:
  • Workload sophistication and complexity
  • Number and size of data sources read by and produced by the workload
  • Data transformation and analytic components of the workload: i.e., creating new derived attributes, combining disparate sources of data, creating descriptive and/or predictive assets such as models
  • Number of downstream users of this workload’s results, and
  • Sophistication score of the user who is the creator/owner of the workload.

Workload, Data and Infrastructure Costs

• Maintains hundreds of terabytes (some in the petabytes) of data files. But 80% of those data files haven’t been read or used in over 24 months, leading to allocation of millions of dollars in wasted high-cost data storage.
• Maintains thousands of workloads and hundreds of thousands of data files. But 15% of the analysts who created those assets don’t actively use SAS, and a whopping 40% of those analysts are no longer employed by your company. That’s a massive exposure in terms of intellectual property and compliance risk. Yet these enterprises don’t know how to find these files and fix the problem.
• Among these thousands of workloads, roughly 30% move data out of your enterprise databases and across the network to local file systems and hard drives without needing to do so. This creates an enormous negative impact on shared resources.

Workload, Data and Infrastructure Risks

• Maintains thousands of workloads. 15-20% of those workloads were authored by analysts who open-text pasted their passwords to critical systems of record in their code. But finding and mitigating these security risk exposures is a needle-in-the-haystack detective hunt.
• More than 40% of those thousands of workloads were authored by analysts who wrote essentially no documentation. Not only does this present a validation challenge, this also creates compliance risk exposures.
• Maintains hundreds of terabytes of SAS data files. Roughly 5% of the records in these files contain Personally Identifiable Information (PII)–names, account numbers, email addresses, tax IDs, etc. This is another substantial compliance risk exposure.

Workload Transformation Potential

• Hundreds of analysts. Among these analysts:
  • 20% are well prepared for analytics modernization
  • 40% need significant direction to get started, and
  • 10% tell us they’re desperately clinging to the tools, skills and practices they learned in the 1980s and 90s.
• Thousands of workloads. Among these workloads:
  • Roughly a third of all SAS workloads pull data out of the database and conduct simple transformations in SAS, then write to BI applications or flat files, but most of this work could have remained in the database, avoided the data transfer impact on the network, avoided the need for redundant backup and lineage tracking, and additional validation.
  • Another third would benefit from being integrated with open source analytics techniques in Python and Spark to save money on analyzing data arriving from cloud sources in highly distributed architectures.
  • 25% would benefit from moving data files from on-premise file systems to cloud object stores like Amazon S3, because the storage of frequently-used flat file data can be more easily stored and accessed there, due to no limits on storage, rather than on finite-sized on-premise file systems.