Enterprise CIOs are often helpless with the facts that they have very limited budget, rare scope for improvement and low resource utilization.  These factors turn IT organizations operating into silos and changing IT infrastructure into “black box”.  So the biggest question of the hour is “How effectively can CIOs consolidate their IT operations and align it with business objectives?”.  That is where the term “Global Service desk” plays very critical role in organization roadmap.

Now the next question – “Why should CIOS have Global Service Desk (GSD)?”.  Technology trends are good; implementing those trends is great.  But are Enterprise IT functions having nightmares with certain perennial concerns? Well, I think so.  Let us check those concerns right away –

a) IT support for every employee within the organization becomes hugely cumbersome

b) Cost of replacing the help desk Engineer is expensive as it includes recruiting, training, and human resources processing costs

c) Building a Service Desk with engineers skilled in supporting multiple hardware and software platforms is always challenging

d) Determining the right number of staff and finding people with the right skills who are willing to work during weekends and night shifts is difficult

e) Low incident/ticket volumes often make it impractical to manage cost vs effective utilization of resources.

f) Multi-vendor Vs. Single Vendor dilemma

Can Enterprises solve the above mentioned problems by setting up a GSD? Well, Enterprises don’t have a choice in the long run.   Service Improvement is the goal of a GSD and it ensures that the IT operations are focussed towards achieving business objectives. The prime objective to set up Global service desk is to aim for exceeding customer expectations with  improved quality of service and reduced cost.  When repeated incidents go unnoticed, it can make a huge impact in all sorts of costs (Resource, communication and vendor).  By taking an integrated approach of ITIL process through Incident Management, Problem Management and Change Management, GSD can significantly reduce these costs. One goal of every business is to achieve the highest possible quality at the lowest possible cost. So the cost and Quality are the two key things which need to be measured on on-going basis.

So How should Enterprises measure the success/ failure of its Service Desk?

a)      Cost per call/email/Incident:  In a Service Desk, the most effective cost metric is cost per call/email/Incident, and the best indicator of quality is customer satisfaction. This is key cost attributed in attending a call/email/Incident by the Service Desk personnel.

b)      Time to respond and resolve:  There is difference between mutually agreed SLAs and timelines vs. actually achieved. First level resolution rate (or) First call resolution (FCR) is the key performance indicator for the Service Desk.  If the Service Desk is achieving a low cost per contact by transferring and escalating contacts to other support levels – Level 2, Level 3, Desktop Support, Vendor Support, etc., then it will dramatically increase the Total Cost of Ownership.

To sum it up, GSD focuses on customer priorities, helping them to enter new markets, adopt new processes, provide better operational flexibility and finally enabling them to serve their clients better.

Faster, better and cheaper is the expected result of any given hour in any given business case.  Now that many organizations have adopted continuous integration in practice, it is now for testers to get aligned to continuous integration development practices. Continuous integration (CI) is a practice in agile methodologies where all developer workspaces are merged with a shared mainline several times a day.

Continuous integration quality control aims to improve the quality of software, and to reduce the time taken to deliver it, by replacing the regular practice of following quality control processes after completing all development.

Continuous integration and its evolution

Continuous integration (CI) was first named and proposed as part of extreme programming (XP). Its main aim is to prevent integration problems, referred to as “integration hell” in early descriptions of XP.

CI was originally intended to be used in combination with automated unit tests written through the practices of test-driven development. Initially this was conceived of as running all unit tests and verifying they all passed before committing to the mainline. Later elaborations of the concept introduced build servers, which automatically run the unit tests periodically or even after every commit and report the results to the developers.

Test Driven Development (TDD) to Behavior Driven Testing (BDT)

A agile in general, is the software development practise to develop the software iteratively.  This is achieved by adding new code at each iteration, but also by refactoring existing code written during previous iterations. This refactoring can be safely achieved only with a strong test system, able to check that the whole software product doesn’t break when new code is added, or when existing ones are modified. Hence, two separate, but strongly connected systems are created:

• The software product that are provided to users
• Its test harness that help build it incrementally

Test Driven Development (TDD) is the best known practice to create this test harness, enabling to create incremental software. TDD is a design technique that drives the development process through testing. In essence you follow three simple steps repeatedly:

• Write a test for the next bit of functionality you want to add.
• Write the functional code until the test passes.
• Refactor both new and old code to make it well structured.

Behavior-driven Development (BDD)evolved to address (TDD) practice issues. BDD really isn’t all that different to TDD. What BDD adds is a clear emphasis on what it takes to make TDD succeed.

BDD starts by identifying business outcomes, and then drills down into the feature set that will achieve those outcomes. Each feature is captured as a “story”, which defines the scope of the feature along with its acceptance criteria.

Behavior-driven Testing(BDT) is a testing approach which evolved from BDD.

1. Develop test scenarios that covers product behavior as in BDD feature pattern
2. Execute test scenarios with manual/automated scripts

Exploratory testing

Cem Kaner, who coined the term in 1983, now defines exploratory testing as “a style of software testing that emphasizes the personal freedom and responsibility of the individual tester to continually optimize the quality of his/her work by treating test-related learning, test design, test execution, and test result interpretation as mutually supportive activities that run in parallel throughout the project

The balancing act: CI, BDT and Exploratory Testing

Balancing Exploratory Testing with scripted behavior driven testing is the way to go.

Though BDT, TDD and BDD help in testing the business outcomes and unit conditions as early as possible in development life cycle, exploratory testing will help in going out of box and catching the interface related testing gaps which we would have missed in TDD, BDD and BDT test script.

• Scripted behavior driven testing – Prepare test scenarios using BDT approach and integrating with UI automation tools for testing inline web & native application.
• Exploratory testing – Reach out of box by executing exploratory testing to ensure test complete.

• Android has captured 68.1% of the Smartphone OS market share in 2012, up from 46.9% in 2011
• From 2.3% in 2011 to 3.5% in 2012, slow and steady, Windows Phone is expecting a major thrust in 2013
• IOS saw a slight decline to 16.9% in 2012 from 18.8% in 2011
• Symbian still hanging on at 4.4% after a steep fall from 16.9% within a span of 1 year

As so many diverse platforms emerging in quick succession, 100% native development for each mobile platform can become a cumbersome, daunting and a costly affair. That’s why when it comes to mobile apps for business purpose, most of the software vendors/enterprises today are deciding on cross platform development. Cross platform mobile app development is a approaching of creating mobile apps which can run across various platforms unlike the habitual mobile apps which run only on the specific platform for which they are developed. With developing mobile computing technologies and number of platforms or operating systems the demand for various mobile computing platforms is distributed among the users. While a few opt for iPhone some may go for Android technology. A demand for Windows mobile is also widespread attracting the attention of several budget technology users. As such businesses today need to target various platforms for getting better business and therefore need an application that can give best possible performance on different platforms to attract the attention of prospective users.

Some of the advantages a cross platform mobile application development process offers are highlighted below:

• Does not need to spend every time to develop a new app for a different platform
• Attract a huge widespread group of audience with a single mobile app well-suited for different platforms
• Do not require to search for different service providers for mobile apps development on different platforms. Efforts for finding reliable service providers every time are eliminated
• Allow organizations to target the largest group of audience (users across various platforms) to promote their app in a better way
• And finally the important factor Cost is lesser than going for development of mobile apps on individual platforms

For any queries pls drop me an email at: archana.ramesh@aspiresys.com

Typically, the software development life cycle (SDLC) is utterly focused on delivering fully functional application software/product on time and within budget. Application performance is often overlooked. But that may be too late for most applications/products to meet current day’s performance requirements and keep up with future expectations.

Application performance should be cautiously considered during requirements gathering, application architecture, design, implementation and deployment. Otherwise, you may end up with a software application/product that has an architecture/technology that is functionally advanced but will not perform well or cannot easily support to changing performance demands.

The ideal way to start performance engineering is together with each phase of SDLC. Based on that principle, the end-to-end performance engineering process can be divided into the following five phase.

• Initiation – Understand the application, identify performance requirements
• Planning – Performance tuning strategy plan, collecting production data, finalize performance objective
• Execute & Analysis – Load test scenarios, benchmark test, analyze test results
• Tuning – Tuning of system, application and database & review

While adopting this approach to performance engineering is the most effective and usually produces the best results.

What is your approach to performance engineering in your organization? Mail me at Archana.ramesh@aspiresys.com

Marked focuses on real user functionality helps match user goals to significantly increase user adoption. This delivers additional value through the following:

• Highly accessible systems for greater accuracy
• Navigable, simple and engaging user-experience for your customers
• Fast and reliable UI for increased user satisfaction

Check out a sample screenshot of an UI Prototype:

For any queries please contact archana.ramesh@aspiresys.com

When compared to a website prototype, the most significant and basic difference is the availability of wide range of devices – the sizes of those devices, the number of characteristics they have and the different operating systems out there. Mobile apps techniques are different as they require the testing of a number of different kinds of interactions which include touch, zoom, and multiple inputs. Animation is another feature which creates a need of different prototype techniques to be available for mobiles. Third is device testing which means that the developers should be able to test the app on the actual device it is built for.

For any queries please contact: archana.ramesh@aspiresys.com

What is a metric? Metric is a standard of measurement. A metric is a measurement criteria that helps measure past and present performance and/or predict future performance.
Most of the Metrics have similar following category of which they are comprised of:
Quality: Purposeful/meaningful measured results of test/execution which represent the product quality. Some of the examples are: Defects logged, Usability, performance, scalability and customer satisfaction.
Progress: Specific parameters which help identify the test progress are compared against the success criteria, are collected iteratively over time which can be later on used to represent the progress in epitomized way.
Coverage: In order to measure the test scope and success, meaningful parameters called Coverage is required.

What is ATM and how good it should be?
Automated Test Metrics are used to measure the past, present and future performance of its process and the relevant efforts been put. A metrics stands good only if it is related to the performance of the effort and it can only happen if there are clearly defined goals pertaining to the automation effort. A well defined automated Test metric has following characteristics:
-Its clearly measurable
-Its purposeful/meaningful
-Epitomized data/Graphical data representation is derived from easy collected data
-It should be valid input criteria for Automation improvement.
-Importantly, it should be simple.

Automation Test Coverage
Automation Test Coverage metrics actually determines via test execution results that whether we have actually achieved what we had covered in past during automation of test cases.
Altogether with manual test coverage, against the total number of tests this type of metric can measure the completeness of the test coverage and can measure how much automation is being executed.

ATC  %    =    Automation Coverage / Total Coverage

Here ‘Total coverage’ means, requirements, units/components, or code coverage.

Automation Progress
Automation Progress points towards the progress of Number of Test cases automated out of the total number of test cases which are automatable.

AP %    =    Number of test cases automated / Number of test cases automatable

The automation progress is tracked over the time, as the automation progresses against the defined Automation milestone tasks, the metric can present valuable data for the time it will take for the whole set of test cases to get automated.

Automation Index
Every project have a sufficient duration, either has automatable test procedures or requires automation from scratch/begin. To fit either of these criteria’s, Automation index or Percent Automatable can be defined:

AI or PA %    =    Number of test cases automatable / Total Number of test cases

Defect Density
Defect measurement is vital for both manual and automated process. If any module/ component of application under test it detected with high density of defects, then it automatically becomes prone to retesting using automation. Defect density is measured using the metrics, the total known defects divided by the size of the software entity being measured.

DD    =    No. Of Known Defects / Size of the Software/Application

Defect Aging
Defect Aging is the difference between – the date when a defect is detected and the current date (if the defect is still open) or the date the defect was fixed (if the defect is already fixed).

DA ( in Time)    =    Defect Fix Date (OR Current Date)   –   Defect Detection Date

Defect Trend Analysis
Defect trend analysis is the trend of defects found over time. This trend truly shows the health/stability of the project. Defect trend is considered to be improving if the no. of defects is reducing over a time, trend is considered to be worse if the defects are increasing. In an agile methodology where the test deliverables needs to be fit in iterative cycles, if the defects count for component/module is reducing in progressive cycles, then it helps in determining the closure of the feature test.

DTA    = No. Of Known Defects / No. Of test Procedures Executed

Defect removal Efficiency
Defect removal efficiency is measured in the percentage of effectiveness of defect removal efforts. This metric if used in collaboration with Automation definitely helps improve the quality of the product. Greater the percentage value, better the quality of the product.

DRE    =    (No. Of Defects found during testing ) /(  ( No. Of Defects found during testing + No. of Defects found after delivery ) )

The report identifies three key enterprise app store trends and recommendations that can help enterprises benefit from the trend:

One is to look to mobile-device management (MDM) partners to include an app store as part of their offering. Enterprise app stores are expected to grow alongside MDM and BYOD policy adoption, and many MDM providers have simple ways of extending apps to mobile devices, as well as to PCs. Gartner advises that organizations begin by assessing their immediate need for an app store and then looking for providers that can include an app store in a mobile management bundle.

Second, enterprise app stores give end users more control (which they welcome) and the ability to choose the applications that best address their needs. “By delegating choice to end users,” said Gartner, “organizations can delegate many important price and performance decisions down to the end-user level.”

Lastly, enterprises can help ensure the success of an app store by keeping it full and vibrant. “Without a dynamic selection of apps to choose from, users will eventually have little reason to continue to visit an enterprise app store,” according to the Gartner report.

Brian Prentice, research vice president at Gartner, commented, “The implementation of an enterprise app store should be seen as a component of an organization’s application strategy, rather than infrastructure strategy. The primary determinant of success is app supply. As a result, application leaders should be given overall responsibility for any app store initiative, but they should work in a collaborative fashion with other teams. The types of apps downloaded and used provide important information as to what types of solutions are of value to each type of user”.

More detailed analysis is available in the report “Enterprise App Stores Can Increase the ROI of the App Portfolio”. The report is available on Gartner’s website at http://www.gartner.com/resId=2325115

For any questions please contact: archana.ramesh@aspiresys.com

A set of guidelines like coding standards, test-data handling, object repository treatment etc… Which when followed during automation scripting produce beneficial outcomes like increase code re-usage, higher portability, reduced script maintenance cost etc. These are only the guidelines not the rules that must be followed, but if one is not following the guidelines then he/she must be losing the benefit of framework.

What are the various Automation Frameworks available?

• Linear Scripting
• The Test Library Architecture Framework.
•  The Data-Driven Testing Framework.
•  The Keyword-Driven or Table-Driven Testing Framework.
• The Hybrid Test Automation Framework.

Linear Scripting – Record & Playback

This is most simple type of Automation framework. In this tester record the test steps and insert some checkpoints and Playback the Test Steps with hard coded values.

Advantages

• Fastest way to generate script
• Automation expertise not required
• Easiest way to learn the features of the Testing Tool

 Disadvantages

• Little reuse of scripts
• Test data is hard coded into the script
• Maintenance Nightmare

The Test Library Architecture Framework

It is also known as Structured Scripting. Tester will create the test script by using “Record and Playback” method and then group the test cases to create Functions, modules or Drivers.

Advantages

• Higher level of code reuse is achieved in Structured Scripting as compared to “Record & Playback”
• The automation scripts are less costly to develop due to higher code re-use
• Easier Script Maintenance

Disadvantages

• Technical expertise is necessary to write Scripts using Test Library Framework.
• More time is needed to plan and prepare test scripts.
• Test Data is hard coded within the scripts

The Data-Driven Testing Framework

In this type of framework, Test case logic resides in the script and test data in kept outside the script, and it is maintained either in Xls sheet, .csv files or in some Database like SQL or Oracle etc. Test Scripts themselves are prepared either using Linear Scripting or Test Library Framework.

Advantages

• Changes to the Test Scripts do not affect the Test Data
• Test Cases can be executed with multiple Sets of Data
• A Variety of Test Scenarios can be executed by just varying the Test Data in the External Data File

Disadvantages

• More time is needed to plan and prepare both Test Scripts and Test Data

The Keyword-Driven or Table-Driven Testing Framework
The Keyword-Driven or Table-Driven framework requires the development of data tables and keywords, independent of the test automation tool used to execute them. Tests can be designed with or without the Application. In a keyword-driven test, the functionality of the application-under-test is documented in a table as well as in step-by-step instructions for each test.

Advantages

• Provides high code re-usability
• Test Tool Independent
• Independent of Application Under Test, same script works for AUT (with some limitations)
• Tests can be designed with or without AUT

Disadvantages

• Initial investment being pretty high, the benefits of this can only be realized if the application is considerably big and the test scripts are to be maintained for quite a few years.
• High Automation expertise is required to create the Keyword Driven Framework.

The Hybrid Test Automation Framework.

As the name suggests this framework is the combination of one or more frameworks pulling from their strengths and trying to mitigate their weaknesses. This hybrid test automation framework is what most frameworks evolve into overtime and multiple projects.

Divide the Test Automation Effort:

Automation task is assigned to Automation test engineer  according to their skill set. it is most important to identify the skill level and experience of your team members. Whether a person is comfortable in which zone. Like for effective script writing a person should have good knowledge of Scripting language. Some team members may not be versed in writing automated test scripts. These QA engineers may be better at writing test cases. It is better when an automated testing tool has a way to create automated tests that do not require an in-depth knowledge of scripting languages, like TestComplete’s keyword tests feature. A keyword test (also known as keyword-driven testing) is a simple series of keywords with a specified action.

You should also collaborate on your automated testing project with other QA engineers in your department. Testing performed by a team is more effective for finding defects and the right automated testing tool allows you to share your projects with several testers

For any Query you can contacy anu.budhiraja@aspiresys.com

The benefit is such that the results are derived earlier than expected; risks are mitigated and thus allowing flexibility for  change.
Agile Testing
Agile Testing is a software testing practice which adapts and follows the agile methodology and considers the development as its customer for testing.

Agile Tester Skills
Agile tester plays major role in automation of agile software under development. Apart from testing skills, an agile tester must possess specific skills mandatory for performing his duties in the Agile test environment, such as:
-  Have deep understanding of the domain, features, development process, design, delivery and deployment
-  Must have skills to understand several stories or features at one time
-  Knowledge on various license or freeware automation tools which suits the business requirement under development

Core Values that forms the Basis for Agile
The thumb rules that govern the agile testing at base level are:
Communication –Communication of Requirements among Developers and testers via quick and small team meetings
Simplicity – Providing simple solutions to things taken first and then move forward with enhancing the features
Feedback- Based on core insight knowledge of the customer, the feedbacks are taken from customer on frequent basis
Iterations- The system is broken down to smaller modules which are accomplished within iterations on which the customer has better control

Automation Test Strategy
Automating the Test in agile development is  very critical decision as agile development is time and iteration based accomplishment which requires decision against specific questions like: When, how and how much to automate? Although Automation continues to follow its base-lined waterfall model but best practices are required to run it in Agile environment.

Initiate Automation First than Manual test?
There are always certain test cases or features which are never prone to defects hence can be directly automated. But before this, a quick level of adhoc / exploratory testing will satisfy the stability of the features hence decision will be taken early to automate them.

Progress Automation with Caution: what to automate- Black box(UI) or White Box?
Automation becomes prone to vulnerability if the importance and usability of the functionality is not identified. Because of frequent changes in the UI design, the idea of automating it could be tedious. Hence for minimal maintenance the white box testing or changes are less fragile and should be automated first with regular coordination with developers where the frequent changes of vulnerable UI can be handled later on.

Automation Start: Coverage based Automation
With effective development practices in place, the automation can generate good results. As the features are delivered under each iteration, the code coverage based automation focuses on building the step by step functional coverage to trace the functional areas which are still pending or remain uncovered. Possibility exists that with the help from developer and customer, tester can develop their own tool to meet the business requirements else if the suitable tool exists in the market which meets the business requirement demands then its wisely taken.

TDD- Development Reliability and Automation benefit
Test Driven Development is one of the initial reliable forms of unit testing practice used by developers which helps them develop production code through rapid iterations. Testers can also follow and create their automation tests in similar manner that it becomes the base for the source code development and hence the following actions can be taken care:
-Code designing at high or low level
-Writing automated unit test cases
-Executing the unit test cases to ensure that they fail
-Re-executing the unit test cases to make sure this time they pass
-Restructure or organize the production code so that it runs better
-Work in coordination with development team to chose right scenarios and implement the same for lower code maintenance and sustainability
TDD helps automation tester verify its own code to make it defect free, robust and reliable

Automation Code coverage review like Manual test cases
The way testers get their test cases reviewed from Developers or business analysts, similarly the Automation code coverage against the features developed in agile iterations also needs to be verified so that:
- Detect issues early
- Preventing the Defect(s) repeated history as the defects cycles are reduced
- It helps in reducing the development and test cycle
- Build strong product/feature knowledge for better stability and design
- Helps make the automation code reliable and robust

What should not be automated:
One time tasks and the exploratory test cases, which probably will never be executed, should not be automated for e.g., configuration and deployment action. Automation of such test cases generally does not get attention or paid off well.

With the above briefs, eventually its clear that automation test in agile environment is very much decision based activity which requires immediate attention towards: What to automate and what to not, which tool should be used, prioritizing the stable and unstable features, code review, communication and feedback. Not only this, I personally think that one of the major breakthrough would be, much improved customized frameworks as more and more coordination will exists between developers and testers to achieve each agile iterations and deliver low risk, high quality products.