Join the BPMN Forum Discussion Group: 600+ members, 580+ topics and growing! Click here to join…

How to Define BPMN Tool Evaluation Criteria for Your Team & Project

The BPMN modeling tools reviewed on this site are evaluated using weighted evaluation criteria. This article describes the BPMN modeling tool evaluation criteria applied, and shows how they can be weighted to customize tool evaluations to meet team and project specific needs.

Evaluation Criteria

The following evaluation criteria separate functional and non-functional tool features. For functional tool features they distinguish between diagram drawing functions , which are can be supported by templates for popular drawing tools (e.g., Visio, OpenOffice Draw, iDraw), from modeling tool specific functions, such as Simulation & Execution functions.

Usability

We evaluate modeling tool Usability while exercising the tool Functional Features described below. Usability factors that we evaluate include, but are not limited to: User Interface (UI) design, UI learnability, UI efficiency (as measured by #UI gestures/function), UI response times, context-sensitive help, and technical documentation.

Functional Features

Functional features that we evaluate are categorized into Drawing features and Simulation & Execution features:

Drawing Features

We evaluate basic, intermediate, and advanced drawing features of modeling tools. Basic drawing features that we assess include, but are not limited to: opening/saving models, opening/saving diagrams, drawing model elements and relationships, editing properties of elements and relationships, and multiple undo/redo capabilities. Intermediate drawing features that we assess include, but are not limited to: support for recursive design techniques, requirements traceability across diagram types, generating relationship tables from diagrams, and automated document generation. Advanced drawing features that we assess include, but are not limited to: model management and tool customizability via profiles, options, and scripts.

Simulation & Execution

Simulation and execution features that we evaluate for BPMN modeling tools include, but are not limited to: simulating Business Process Diagrams, enforcing constraints and well-formedness rules, and code generation.

Standards Compliance & Interoperability

We evaluate tool compliance to BPMN notation (syntax) and semantics as defined by the current BPMN 2.x specification. We also evaluate the interoperability of XMI files generated for model import/export, and the interoperability of BPMN diagrams with UML diagrams for modeling projects with Business Analysts and Software Engineers that require mixed language usages.

Technical & Team Modeling Support

We evaluate general tool support to include installation, set up, update mechanisms, and technical support.  We also evaluate team modeling support for project sharing, versioning, and user permissions.

Value

We calculate Value as a function of Feature (Completeness,Quality) / Price in order to rank tools in relative order.

Weighting Criteria

After you have specified your selection criteria you should assign numerical weights to them to quantify their relative importance. The assignment of relative numerical weights to your criteria allows you to tailor your tool selection to specific team and project needs, and it can also reduce evaluator bias. For example, consider weighting the evaluation criteria described above as follows:

\(Score_ {tool} = \overline{x} = \sum_{i=1}^{n} w_i * c_i\)

where Tool Evaluation Score (\(Score_ {tool}\)) is the weighted mean (\(\overline{x}\)) of the following six (6) evaluation criteria (c1 = Usability, c2 = Functionality: Drawing, c3 = Functionality: Simulation & Execution, c4 = Standards & Interoperability, c5 = Technical & Team Modeling Support, c6 =Value) with corresponding weights (w1 = 15%, w2 = 15%, w3 = 15%, w4 = 15%, w5 = 10%, w6 = 15%).

The weights that you apply to your own BPMN tool evaluations should be adjusted for your team and project modeling needs.

Comments on this entry are closed.