Posted on: Tuesday, November 1st, 2011 by Paul Salasznyk
By: Paul Salasznyk, Ph.D
Last week, with support from Computech, the Comisión Federal de Telecomunicaciones (COFETEL) launched MiCOFETEL, a website that collects and disseminates telecommunications information for consumers within Mexico. The site also serves as a means for COFETEL to assess the status of telecommunications within the nation. Computech’s contribution to the interactive portal focused on broadband, a topic that we have become intimately familiar with during our work on the FCC’s speed test, National Broadband Map and various broadband-related data analysis projects.
Data is the foundation of the MiCOFETEL website. Of paramount importance was Computech’s ability to familiarize itself with Mexico’s geographic and broadband landscape, shape relevant broadband data into a usable format for analysis, design a means to collect important data from consumers (data that was not readily available), and exhibit the data using visualizations. These steps exemplify Computech’s standard data analysis methodology, described in more detail in the detailed steps below.
Data Familiarization
- To gain an understanding of the broadband landscape within Mexico, Computech learned about broadband providers and technologies in the nation, read published broadband research on broadband and studied available broadband datasets.
- To perform analysis for particular geographies within Mexico, we familiarized ourselves with Mexico’s geographic composition (state, municipalities, etc.) and the relationships they have with one another.
Standardizing, Coalescing, and Analyzing Datasets
- To establish a standardized data format, Computech transformed millions of data records on broadband pricing, speeds and complaint data into a geographic “common denominator” for each provider and geography. Computech then extracted useful insights from this standardized data.
- To highlight relationships and synergies between similar (but different) datasets, Computech merged and analyzed data from disparate broadband data sources.
Crowdsourcing Data
- To collect other required broadband data, Computech created a portal that collects broadband data from consumers, including location, provider, pricing and speeds which Computech then analyzes and visualizes.
Data Visualization
- To highlight the analyzed broadband data, Computech created a series of charts, graphs, maps and tables that most appropriately exhibit the analyzed data.
- To improve each consumer’s experience with the visualizations, Computech added interactive functionality, allowing users to filter for various criteria and compare broadband data to other providers and geographies.
Computech’s contributions to the MiCOFETEL site are anchored by its expertise with data – from familiarization to analysis and visualization. Going forward, Computech will work with COFETEL to incorporate additional data sources and improve functionality of the portal, to further increase Mexican consumer understanding of broadband within their nation.
Posted on: Tuesday, October 18th, 2011 by Paul Salasznyk
By: Paul Salasznyk, Ph.D
Yesterday, the Comisión Federal de Telecomunicaciones (COFETEL) launched a website (http://micofetel.gob.mx/) that displays a wide array of information about the Mexican telecommunications industry. COFETEL, which regulates, monitors and promotes the efficient development of telecommunications throughout Mexico, tasked Computech with the construction of several key portions of the new website, including a speed test application and several maps and visualizations of telecommunications information. Here is a summary of Computech’s contributions to the MiCOFETEL website:
1) Speed Test Application and Data Visualizations
In order to provide Mexican consumers with a measure of their broadband connection quality, Computech worked with online speed test provider Ookla and developed a broadband speed test portal. (http://www.micofetel.gob.mx/medidor_de_velocidad) Highlights of the speed test portion of the website include:
- A report of download speed, upload speed, and latency of each internet connection (speed test) on the website as well as IPhone and Android Applications for mobile devices.
- A place for COFETEL to collect location, broadband provider and price information from consumers
- Estimates of download durations (for a online book, song and movie), based on a consumer’s download speed
- Average broadband speed information in the area where a consumer runs the speed test, as well as speed information for other providers in their location
- A summary page that highlights broadband speed test information for each state, municipality and provider within Mexico. Metrics include number of tests, download/upload speeds, latency and metrics by time of day and day of week.
2) Telecommunications Maps
In order to summarize complaint data collected on the MiCOFETEL website as well as broadband provider pricing, and broadband speeds, Computech developed a series of maps. (http://micofetel.gob.mx/mapa_telecomunicaciones) Highlights include:
- A map that visualizes the number of complaints (mobile phone, local/ long distance telephone, television, internet) by state
- A map that disseminates information about average download, upload, latency for all providers by state
- A map that highlights provider availability by state, as well as speed tiers and prices offered by those providers
The launch of the MiCOFETEL site promises to provide an opportunity to gather even more data from consumers in Mexico. Computech is looking forward to working with COFETEL to include additional functionality and datasets into future releases of the MiCOFETEL website.
Posted on: Monday, September 26th, 2011 by Darren Vandergriff
Last Wednesday, the NTIA and the FCC released the first major update to the National Broadband Map since its highly acclaimed debut in February. Computech has been a driving force on the project and has provided application development, GIS, operations research, information architecture, user experience, interaction design as well as expertise in design, front-end development and data analysis services in support of the launch and update. Along with updated data from a larger number of broadband providers, the map also provides additional features and functions.
Below is an explanation of the new National Broadband Map features:
• Data Refresh — We receive SBDD broadband data from all providers throughout the country every 6 months. The site is now updated with data from December 31, 2010 including Search, Analyze and Maps.
• There is a new page that summarizes broadband statistics for each provider in the US. The user can now select a specific geography then view a provider’s availability within that geography.
• We have introduced the sub data set that cannot be found anywhere else.
The user can also compare the selected provider with all the providers in this geography by population served, technologies served and most common download speed.
• There are new maps in the map gallery, including:
Community Anchor Institution Map — Allows a user to enter their address and view the closest 25 Community Anchor Institutions as well as information about broadband at those locations.
Feedback Map — The NBM collects crowd-sourced feedback about the accuracy of the information returned to the user in Search. A new map has been added that plots these feedback points by Provider confirmation and Speed confirmation.
• Several Summarize and Demographic APIs now return new information.
• This release included 10 more APIs. 8 APIs provide information about the providers.
• We have introduced the new API Developer pages which document the APIs more clearly
• We have also included a set of dictionary APIs to help the developers explore all the parameters in the API call and response.
• A brand new Social Layer was integrated which will enable closer Facebook, Google+, Linked in and Twitter Integration. Facebook integration now includes geospatial open graph tags. As a results of this integration, The National Broadband Map is already appearing in the Facebook Search results as a unique result. We have taken the opportunity to associate all pages of the site through National Broadband Map geographical keywords into the Facebook search.The number of modules that power the National Broadband Map has increased to 1050.
According the National Broadband Map blog, the Map is used by business owners, consumers, policy makers, research firms, academics and application developers, just to name a few. In its first seven months, the National Broadband Map has received millions of page views and countless amounts of praise.
Posted on: Thursday, July 28th, 2011 by Corey Jerome
Federal Financial Modernizations Should Limit Customization
When government agencies procure and install a new core accounting system, their goal is to limit customization of the software and adapt business processes to function within the new system. However, some processes are unique to an agency and must continue to serve their intended purposes. When the business logic clearly does not involve accounting transactions, the decision to maintain a separate system is obvious. However, it is sometimes unclear whether an accounting-related function can be incorporated into a core financial system or whether it should remain separate.
This guide illustrates which functions should and should not be included in the core accounting system.
Functions to embed within the core financial system
It is typically recommended to implement the main functional accounting areas in the core system, including: General Ledger, Receivables, Payables, Payroll, Budgeting and Forecasting, Reporting, Reconciliation and Transaction Journals.
Functions to build and maintain outside of the core financial system
Web systems used by the public
Some accounting systems may include public-facing modules in order to, for example, display invoices or bills or to collect online payments. However, agencies which are committed to providing their users with a seamless, branded web experience, including single sign-in for all web interactions, should develop these systems within the agency and then build the necessary interfaces with the financial system.
Intermediary web services
For any interface to the core financial system which is used in multiple places in the agency, build an intermediary web service with fail-over handling. This will be more efficient and require changes in only one place if the financial system’s interface specifications should change. Additionally, the core financial system will not be available 24/7; it may at times be necessary to queue requests until the system is available and this is best managed in one location.
Processes that combine accounting and non-accounting information
When business logic includes information that exists both within the accounting system and without, the entire function should be built and maintained outside of the core accounting system. For example, certain conditions may prevent the agency from providing service to an outside entity, including the presence of unpaid delinquent debt. Although information about debts are inside the core financial system, other information, such as a pending request to waive the debt, may be tracked in a separate workflow system. The process can easily pull data from both systems.
An extensive collection of custom data fields
Extensive data that is needed to connect revenue or expenses to business processes may be collected and tracked outside of the core financial system. If the financial system cannot easily handle the number of data fields or data relationships that the agency tracks, or the level of detail would produce a burdensome number of accounting lines, then the agency should consider maintaining the detailed information in the originating system and generating a unique identifier, which would then be referenced in the core financial system.
Functions to include in the core financial system and to stand-up separately
Data validation functions
Although data validation must be built into the core financial system, data should also be thoroughly validated before integrating.
Dashboard of integration results
Operations staff should be able to view data being loaded into the system from other coupled systems. They should be able to review this information both outside and inside the core financial system to independently verify that all operations were successful.
Real time information services
If there are dependencies which delay information from reaching the core financial system, the data might not be available as quickly as possible. For example, credit card authorizations may be known in real-time but cannot post as accounting transactions until the funds are deposited. Other actions may be pending entry or approvals. Staging such information outside the core accounting system may make sense in order to accelerate dependent business processes.
Risks of over-customizing the core financial system
Over-customizing an off-the-shelf financial system will likely have adverse effects. The more extensive or complex your customizations are, the greater the risk of a mismatch between business rules and the actual implementation. Agencies can isolate and resolve the impact of such issues more easily within a separate custom system compared with a heavily-customized financial system.
Coupling agency processes too tightly will also lead to additional effort and cost when the agency upgrades the base software package or when it needs to alter businesses processes. Loosely coupled systems connected via web services are going to deliver the greatest value and flexibility.
An agency’s financial department may also experience decreased staff productivity or encounter reduced system performance if the agency insists upon using the system in ways not originally intended. For example, if a complex workflow or a large data set is tying up system resources, then staff will waste time working in a sluggish system.
Agencies face many decisions when standing up a new system; if you are involved in such a project mull over your various options and keep in mind this mantra: When in doubt, leave it out!
Posted on: Monday, July 25th, 2011 by Hariprasad Donthi
When it comes to Data Management, it’s impossible to predict the questions the users will ask of the data. Therefore, it’s better to build platforms instead of hardcoded/hardwired applications and systems.
The point of a Platform is that it is Application Independent. A platform, like the Interstate highway system or an Electric Grid, is meant to be used for things that cannot be specified or predicted when the platform is being built.
It’s the same with a platform for Data and Analytics. It should never be built around specific reports or use-cases.
In almost all cases, agencies store data in application-specific structures.
Complex Applications need to store a lot of data to deal with short-term workflow, e.g., as users go through the process of filling out a form, and the Application goes through the process of fulfilling the requests etc. Most of this data is of no long-term value and is very irrelevant, confusing, and overwhelming for someone looking for patterns and trends, and who is in the business of steering an organization.
The point of Data/Analytics platforms are to remove all of the messy data that is only of transient value to the workflow.
The next debate we usually get into is how to organize this data for a decision-making platform in a normalized structure, dimensional structure, how much to snowflake etc.
These days, most organizations that depend on data/analytics for their survival don’t agonize over the data modeling. This is because of two factors:
1. There is an entirely new marketplace of products that are meant specifically for analysis – for example, column-stores, MPP solutions, distributed-key-value pairs and cloud-stores.
2. New types of data are being generated so quickly, that data models have to be very generic. The Semantic web concept of storing all data in a subject-predicate-object is an extreme example of this movement. Data in science experiments is in totally new structures as people do new types of experiments.
Data-driven companies don’t bottleneck their analytics while the development team conforms incoming data into existing facts and dimensions.
The moral of the story is this: if you want to be a data-driven organization, get the data to the hands of the analysts as quickly as possible, make it clean and understandable, and create a culture where analysts can show-off their results and start discussions.
Standardization of data elements and concepts emerge out of the actual analysis on new data, rather than in specs that are used for a long software development lifecycle. Standards (i.e., Master Data / Metadata management) are still critical, but should not be a pre-requisite for analysis.
« Previous Entries Next Entries »
