IP Infrastructures: Phases, challenges, and technical keys in RTVE’s project in Sant Cugat
By Jesús García Romero, Technical Director at TVE; and Miguel Ángel Corral Toral, Head of Facilities, Engineering and Studio Support Projects and Mobile Units at RTVE.
RTVE’s Sant Cugat studios face an unprecedented challenge in the Spanish television scene: completing the transition from SDI technology to IP technology. Like any migration process to new technologies, it has been necessary not only to face the uncertainty regarding maturity of technology, but also to generate an environment of trust among the operating and maintenance personnel for the systems, who are probably facing the most outstanding technological change they have encountered in their professional careers.
The first of these challenges is overcome by means of exploration and analysis of the market, training, and technical support from ‘those who know best’. The second challenge, dealing with a deep professional retraining of the staff to help them leave their ‘comfort zone’ and ‘change mindset’, requires a more profound restructuring. However, we can state that the attitude of the entire technical and operational team at Sant Cugat has been impeccable, adopting a highly receptive position right from the outset and providing a level of collaboration that is undoubtedly having a highly positive impact on this transition’s success.
A bit of history
The project was born from the Television Technical Area Directorate when the time came to approach migration of four studios in Sant Cugat to HD. Consideration was made of the convenience of taking advantage of the moment of change to ‘go even further’ towards a technology that, although apparently not mature enough, it was felt that it would not take long to become a solid reality in chronological parallelism with the time necessary to prepare a tender, whose concept was far from what had been developed so far.
The decision was clear. We set out to ‘get on the bandwagon of what’s to come’ rather than continue to exploit established technologies. We would do the latter with very good results, but the truth is that in a short time these technologies could already become outdated. A sizable investment must be accompanied by an assurance of validity over time of the technology for which a commitment is made. And to successfully achieve this goal, nothing better than to be guided by the advice of one of our golden age writers, Baltasar Gracian:
‘It is beneficial sanity to save yourself trouble. Prudence avoids much of it.’
And, as good things come to the prudent, they proposed a first approach to the IP world that they had already seen at the IBC and NAB, Bit Media, BITAM, etc. professional trade shows, especially since they could physically interact with equipment models in operation offered in the very interesting IP showcase of these fairs. In May 2018, RTVE’s Operations Department convened the first workshop on ‘Transition of broadcast technology to the IT world’ in the Prado del Rey auditorium, attended by industry professionals of the highest level. It became a reference as it was the first approach to take place in Spain aimed at driving the certainty that the transition from the broadcast world to IT technology was already a reality.
With such certainty, a small engineering team traveled to the legendary Pinewood Studios in London. There, from the hand of SONY, they received an interesting theoretical approach to the IP environment and got to know an initial Live IP experience.
“A remarkably interesting IP experience, also very galactic-like, because the shooting of a movie from the Star Wars saga had just finished and among the remains of the sets was the Millennium Falcon’s cockpit. This was a sign that the Force was with us in this project.”
During the following months, the TVE engineering department worked in the same direction, with a single goal in mind: to deepen the knowledge of IP technology through intense specific training on networks and IP video standards, throuogh numerous meetings and concept tests with integrators such as Telefónica, Unitecnic, Crosspoint or Qvestmedia; engineers from broadcast manufacturers such as Abacanto, Nevion, GrassValley, Sony, Lawo, Xeltec, Calrec, Evertz, EVS, Axon, Sapec or Net Insight; from IT manufacturers such as Arista, Cisco or Melanox; and orchestration and control software developers such as VSM, Cerebrum, Orbit or BFE.
As part of the information gathering process, three overseas trips were taken to IP video facilities. This is how contact with real experiences was made: the Proximus production centers in Belgium or the giant NEP in the Netherlands; and with the help of Sony and Telefónica Servicios Audiovisuales, a visit was paid to see the IP implementation that these companies had carried out on Portuguese television SIC in Lisbon. In these visits, whatever concerns had been generated during the assimilation of new knowledge were conveyed. It could be seen how the path had been difficult for those who had done it before, but the goal was not an unattainable one. RTVE took these experiences into consideration as advantages for their project’s design: they served to properly address certain aspects of the project learned from the experience that others had gone through.
After this entire journey, an RFI (Request For Information) was drawn up and sent out to all companies -mainly integrators- of relevance and solidly established in the broadcast sector. For the first time in a broadcast-oriented way, the main IT brands joined forces, and they had a lot to say in this new journey. With their unstoppable evolution in increasing bandwidth management capabilities, they have positioned themselves as the alternative and foundation of the new video signal transmission systems. At the same time, and to complement the fact that all the signals would travel through the network, a new type of player arose: companies dedicated to the development of orchestration systems, capable of offering a layer of control over all the equipment that was unheard of up to then. This would add agility and flexibility to the systems in aspects relating reconfiguration and new adaptation to different working modes.
Taking on this challenge generated an avalanche of proposals for participation. After a period of two months that was needed for working in depth on the requirements, exhibitions arrived along with proposals at the highest level. This only strengthened the idea that broadcast technology was unstoppably advancing towards convergence with the IT world, consolidating the governing standards to come around this goal; a roadmap that the JT-NM published with very specific dates for the achievement of well-defined milestones.
Inception of the project
After 2 years from the beginning of the process, in October 2019 RTVE published the first tender that contained a proposal for a complete IP video and audio installation for 4 television studios in its Program Production Center in Sant Cugat and a remote set in the city of Barcelona. Said specification also contemplated compliance of the new IP installation with important ‘legacy” systems, such as the signal recording system currently in operation for 6 studios and the exchange of a hundred signals with the Central Control matrix. All this, in an installation designed for HD-3G and dimensioned for redundancy, both for the 4 studios that were the subject of the contract, as well as for future compliance by the other 3 HD studios and the Center’s continuity system.
This project was proposed as a complete, encapsulated, high-definition digital audio and video technology installation for transport over IP technology for the production of four studios with a recording system to which three other independent items were added: a virtual set, a set in the newsroom and a remote set.
The video was based on 1080i-format high-definition serial digital video, in an installation that had been considered for 1080/50p format with HDR; and to allow future compliance with IP technology by studies 5, 6 and 7. Audio was treated as PCM digital audio encapsulated in IP (AoIP) under the AES67 Profile B standard up to the audio mixer’s output in order to minimize latency. From there onwards Profile A would be used for audio as it was compatible with 2110-30. In this way, audio and video would have as final signal encapsulation format the one corresponding to the SMPTE 2110 set of standards (Professional Media Over Managed IP Networks), a contribution by the Video Services Forum as a set of technical recommendations for the transport of elementary media streams over IP networks.
This new technology adopted by RTVE for the new Sant Cugat studios offers three key points that improve upon the current coaxial digital video baseband infrastructures; namely:
a) Simplicity in installation, a significant decrease in wiring and resources (distribution, processing …).
b) Versatility, by allowing the software to assign various functions to the hardware. And there is a much greater density of signals when a large number of essences of different nature are conveyed through the same link (the network is signal-type agnostic).
c) Virtualization, as cameras, mixers, signal processors, gateways, etc. turn into generic, readily re-assignable resources. Other advantages arising from these are ease of migration to the new booming formats and ease of expansion and/or adherence of new studios to the system, which had already been foreseen in the initial dimensioning of the IP infrastructure.
This novel approach also allows and promotes sharing of resources, being this feature adopted in the installation not only regarding reallocation of media between studios, but also in the aspect of electronics sharing, which then becomes centralized. Such is the case with the electronics of the video mixers in the studios, for which a proposal was made to acquire only one for every pair of studios along with two remote panels for the production controls, for instance.
In this way, preparations to carry out special programs that previously required complex procedures for transfer of resources or which simply could not be carried out at all, are now possible thanks to this technology with a shorter intervention time, thus increasing productivity of the facilities. On the contrary, in the initial phase installation commissioning times increase, as complex programming tasks are required during the new systems’ start-up.
The studios receive a signal and can remote the referred local and distant sets by relying on the necessary technology to produce, in certain cases, by means of virtual reality. Installation of synchronisms has also been expanded with the addition of the PTP signal and the intercom system, including RRCS for communications management.
In addition, two new tools not present until now in the usual installation of studios have been added:
– An orchestration and control system (‘broadcast controller’) that governs studio equipment and network electronics, thus allowing to quickly and flexibly make configuration changes, schedule events or plan for different work modes via reallocation of resources, enhancing, for instance, the resources of one studio versus another in the undertaking of large productions.
– A management, monitoring and alarms system that offers operators a global view of the running processes and the status of the network and the equipment, with interactive notices that allow reacting to issues or anticipating congestion situations that may occur.
Telefónica Soluciones is the integrator for this project together with the TSA technical team. It also relies on the technical advice of the engineers from the distributors: Crosspoint for Grass Valley and Embrionix/Riedel equipment; and Xeltec for the entire range of Lawo systems. The main brands that Telefónica Soluciones relied on for its batch as integrator were:
– Cisco for network electronics. The product is IPFM (IP Fabric for Media) with a key element, a development effort named NBM (Non-Blocking Multicast). It also provides the network electronics manager, which is Cisco’s DCNM.
– Lawo for gateway processors, for multiscreen generators and for the VSM orchestration/broadcast controller system. Also part of VSM is the set of monitoring and control tools: vSNMP, SmartDASH, SmartSCOPE, theWall.
– Grass Valley for Kahuna 9600 video mixers with Maverik panels.
– Embrionix for monitoring gateways in controls.
– Stands, robotic heads and Vinten robotics control system, CUE Autoscript signal generators and BlackMagic Ultimatte 12 virtual background embedders.
Generators hooked to PTP GPS systems and traditional Tektronix SPG8000A synchronisms, as well as Tektronix PRISM WFMs for measurements in the IP world.
Classic synchronism distributors (BB, TriLevel, DARS…) with equipment from the firm Albalá Ingenieros.
Crosspoint is the successful bidder for the Camera System batch, which comprises GrassValley LDX 82 Elite, and offers a complete system with CCS-One as an access point for the configuration of the system by VSM.
On the other hand, Telefónica Servicios Audiovisuales is the supplier of the audio system batch consisting of 4 LAWO mc2 56 MKIII HD consoles and a NOVA Router 73 HD audio matrix (also from LAWO); apart from the LAWO distributed input and output boxes A_Stage64 / A_Digital8 and 4 independent external DSP modules with processing capacity of up to 256 channels each (A_UHD Core).
The execution phases for this project were the following:
– Release of the call for bids (October 2019)
– Contract award (March 2020): Creation of a ‘technical office’ for project management and field measurements in order to plan for the installation and commissioning of orders / procurement of materials.
– Base architecture installation (May 2020): Installation of network infrastructure and equipment for a ‘pilot’ study.
– Pre-training (June 2020): Aimed at providing the company’s professionals with theoretical foundations on these new technologies. Also oriented to the acquisition of knowledge about “what the system can do”, especially regarding setup of the orchestration and control system, allocation, and management of resources; and design of software panels representing RTVE’s own workflow.
– Installation of Production Studios (September 2020): Once the infrastructure works carried out on the controls for studios 3 and 4 (July and August) were completed. Meanwhile, production was maintained versus the same sets from a mobile unit located outside.
– Network and systems configuration: Cameras / Mixers / Gateways / Multiscreens (September and October 2020): Commissioning and setup of Cisco, Grass Valley, Lawo and Embrionix systems, pursuing the interoperability of them all as a whole.
– Approval of the pilot study (November 2020): Demonstration of the degree of interoperability achieved between the equipment and the orchestration system.
– Training (October 2020 to February 2021): Training courses regarding all areas involved in the project, concerning equipment and systems operation and maintenance, acquisition of skills on management of the orchestration system and familiarization with management tools for network electronics.
– Go live – Production Studios (March 2021): Commissioning of the first pair of studios (studios 3 and 4 for Program Production).
– Installation of News Studios (June 2021): Once the infrastructure works carried out on the controls for studios 1 and 2 (July and August) are completed. In the meantime, production is maintained versus the same sets from a mobile unit located outside and/or from the controls of studios 3 and 4, already in operation, whenever the timetables of the relevant programs do not overlap.
– Go live – News Studios (July 2021): Commissioning of the second couple of studies (News Studies 1 and 2).
– Installation of remote, virtual sets and newsroom (July/August 2021): Installation and commissioning of sets close to the studios.
As we have seen, we are facing a complete IP video and audio installation for 4 television studios at the Barcelona Program Production Center. Furthermore, also contemplated is compliance of the new IP installation with important ‘legacy’ systems, such as the signal recording system currently in operation, 3 EVS XT VIA servers shared by the 6 studios and the exchange of a hundred signals with the Central Control SDI matrix. All this, in an installation designed for HD-3G and dimensioned for redundancy, both for the 4 studios that were the subject of the contract, as well as for future compliance of 3 additional studios (studio 7 for news programs and studios 5 for program production) as well the Center’s continuity system. But what specific elements do they comprise?
1. NETWORK ARCHITECTURE
The design of the media network is based on a network architecture featuring SPINE & LEAF topology (with a single SPINE), comprising three LEAF nodes connected to a central SPINE. These nodes correspond to equipment for News studios 1 and 2, equipment for studios 3 and 4 for Program Production, and connectivity equipment with the legacy of the current SDI installation based on gateways (shared resources). The SPINE and the first two nodes are located in a ‘full IP’ Equipment Room in the Studio area, whereas the third node is located in the Central Control Equipment Room found in the Technical Block. Likewise, all the above-mentioned nodes are duplicated, thus making up two completely independent networks: the red network and the blue network.
Said network electronics is covered by Cisco equipment, being the SPINE and the Central Control node stand-alone type switches, each of them featuring Nexus series 36 100G ports. The electronics of each of these two nodes of the chassis studios offer capacities of up to 4 line cards, of the same family. As mentioned before, each of the electronics is duplicated for both networks.
The network solution provided by Cisco is called IPFM (IP Fabric for Media), based on the use of transport at level 3 (IP) using the multicast protocol. Cisco adds to the multicast transport protocol (PIM) its NBM (Non-Blocking Multicast) development, a process under NX-OS (operating system used by the switches of the Nexus family), that provides an extra intelligence to PIM for the use of efficient links between nodes. DCNM is the network manager.
The audio network is practically independent from the control network, except for the fact that in very specific locations (GCL in the studio area) switches share audio and control signals, being therefore connected to the cores of both networks. In the rest of the audio installation, the switches are dedicated, thus forming two parallel networks (red and blue networks), with their relevant cores installed in the IP Studios Equipment Room (studio area) and connected to the SPINE of media networks so as to enable audio-video alignment in studio production.
In media networks (video and audio), the SMPTE 2110 video over IP encapsulation format is used.
There is a synchronization network based on PTP comprising two separate, dedicated switches for this type of signal, operating in ‘transparent clock’ mode and installed in the Central Control Equipment Room. From them, the master PTP is sent to the SPINEs of the media network; and, from this entry point, the PTP is distributed to all the media switches working in ‘boundary clock’ mode. The switches in this network receive a signal from two Tektronix SPG8000A generators connected to GPS, also located in the Central Control Equipment Room. The PTP signal is generated under a single PTP SMPTE 2050 v2 domain, as there is initially no equipment in the installation that requires working under another PTP domain.
The design of the control network conforms to a ‘pseudo SPINE-LEAF’ architecture, in the sense that there are not two parallel and duplicated networks, but a distribution of the signals by zones towards the LEAF switches, each one of them connected to both SPINES, thus setting up a double path for the control signals instead. As in the previously described networks, the cores (SPINES) are installed in the Equipment Room of the Studios, and the LEAFs are distributed by zones for convenience for the aggregation of equipment. A LEAF is also deployed in the Central Control Equipment Room.
Last, it must be mentioned that there is a network of cameras -separate from the ones previously described- that is not directly connected to the SPINEs corresponding to the media network, but to the CCU elements uploading and receiving video and audio signals in SMPTE 2110 format from said network. This network is made up of Cisco switches also from the Nexus series.
Within the camera network, we work with the SMPTE 2022/6 video over IP encapsulation format. The main purpose of this network is to enable block exchange of all signals between camera heads and CCUs. This is a particularly good approach to working with the Remote Set, so that signals are exchanged with this set without causing ‘misalignment’ between them.
Future growth of the networks
The architecture for the media network of this facility encourages an ‘east-to-west’ growth through addition of new LEAFs, with the limit of connectivity to the SPINE being determined by the number of free links available.
For this reason, an expansion strategy will either involve adding new LEAFs by connecting them to the SPINE whenever there are links available (100G links); or expanding instead the capacity of the SPINEs, in which case they would be replaced by SPINEs having a larger number (and where appropriate size) of ports, the then released SPINEs onwards taking the role of new LEAFs, which results in some of the equipment being reused.
All current links between LEAFs and SPINE have been calculated:
– By considering for each 1080 50p video streams workload of 2.5 Gbps (actual load: 2.15 Gbps), operating in fact, as already noted above, in 1080 50i.
– By considering that the number of current links referred to each LEAF includes a SPARE link (not necessary in principle due to load volume).
– By assuming that 15% of ports of each type are free in every LEAF.
– By taking into account all the dimensioning of future studios. In this regard, 3 additional links to those currently connected will be used when the time comes to join the system for these studios.
The control network is also expandable without any further trouble by simply by adding new control switches that increase the control ports of the new equipment and are connected to both control cores.
2. GATEWAY PROCESSORS
HD-SDI/IP/HD-SDI gateway-type conversion systems implemented in Barcelona are sorted out by using various products existing on the market based on their relevant role in the installation, as we find the following environments:
• High-density gateway processors: used for the legacy signals associated with the Central Control matrix and for the upload and download signals versus the junction boxes found in the set. C100 Lawo software-defined processors mounted in the v_Matrix enclosure are used and connected to the red and blue media networks via 40 GbE interfaces.
• Mini-module based gateway processors: used for discrete monitoring in controls and sets, featuring HDMI or SDI outputs. Converters by Embrionix, a brand recently acquired by Riedel, are used and fitted in small boxes and connected to the red and blue networks with 10 GbE interfaces.
3. MULTISCREEN SYSTEM
Generation of multiscreen signals is based on the same Lawo v_Matrix platform that is used for the gateways. Only that in this case the C100 processor modules that carry out this work come with a multiscreen license loaded and the rear adapted to the function to be performed. There are 19 modules of this type, licensed under vm_dmv, that offer the following processing capabilities (both simultaneously):
– INPUT STAGE: Generation of mipmaps (versions of input signals in various resolutions) for up to 16 external HD-SDI signals, or up to 24 signals present in the network in different formats, provided they do not exceed 40 Gbps in aggregate. In our project up to 24 network signals encapsulated in 2110 1080/50i. The mipmaps generated are available on the network for subscription.
– OUTPUT STAGE: Generation of up to 4 heads (multiscreen outputs to the network generated from mipmaps present on the network) in 2110 1080/50p format with up to 64 sources each (in our case limited to 25 for ease of use, not due to hardware restrictions). The sources may have been generated by the module in question or by any other module.
The 19 modules are divided into two clusters, one for each pair of studios (production or news) along with their associated technical controls. Each cluster processes the signals from their own studios, in order to minimize the exchange of flows between LEAFs, and thus not unnecessarily overload the links between LEAFs and SPINE. Connectivity to the media network is carried out through 2 40 GbE interfaces (one for each of the red and blue networks).
4. CAMERA SYSTEM
The camera system comprises a pool of 23 XCU Enterprise UXF CCUs and 30 LDX-82 Elite camera heads, all of them from Grass Valley. Four of these camera heads are integrated with the StarTracker system from manufacturer Mo-Sys for absolute positioning tracking used for virtual reality and augmented reality.
Between the camera heads and the XCUs, Grass Valley Direct IP technology is used for the IP transport of video, audio, intercom and signaling, through the previously described camera network. They also allow an Ethernet trunk between the camera head and the relevant CCU. This allows connecting the robotics control data on the camera head. Furthermore, in its Direct IP+ version, this technology is used in four of the camera heads located at the remote set in Barcelona in order to convey said signals in JPEG2000 compressed format through the NetInsight Nimbra network.
The entire set of camera chains is controlled by the Orchestration System through CCS-One servers in order to have absolute flexibility and to be able to manage the assignments of the camera heads deployed on any set to any production control at convenience. based on operational needs.
5. SHARED VIDEO MIXERS
There are two 72-in x 42-out Kahuna 9600 video mixers with 5 M/E banks each. Each mixer provides service to a pair of studios and therefore there are 2 Maverik panels connected to each set of electronics (4 panels in total).
Therefore, given the electronics shared by each pair of studios, resources between each pair can be balanced so as to enhance at a given moment the resources allocated to one studio to the detriment of the other studio, all of this by governing the mixers from the VSM orchestrator.
6. MONITORING AND SYNCHRONISM GENERATION SYSTEM
Two Tektronix PRISM waveform monitors are available, which allow an analysis of IP signals traveling through the network.
The synchronism generators are also from Tektronix, model SPG8000A. There are two due to redundancy reasons; both generating PTP sync signal hooked to GPS and classic sync for signals outside the IP environment. The plan is to have all RTVE Sant Cugat synchronisms depend on these new generators.
Specific Nimbra cards with concrete SFP modules have been purchased for the exchange of 2022/6 flows between sites and for the transmission of robotics control data and tracking for positioning of sensorized cameras for such purpose.
An Artist 1024 box has been purchased with a card licensed for 110 two-way audio streams (8 channels per stream) and redundant connection to the red and blue audio networks. For intercom audio exchange with audio mixers (16 streams per studio) and with camera CCUs (46 streams, production and engineering, for 23 CCUs).
9. CONTROL AND ORCHESTATION SERVERS
Installation comprises several servers to service Lawo’s orchestration and control systems such as VSM, vSNMP, SmartDASH, SmartSCOPE as well as VSM control hardware and software panels. It also has two servers for Cisco’s DCNM system dedicated to network management: DCNM.
Lawo’s VSM (Virtual Studio Manager) is the system of choice as broadcast orchestrator and controller. The system controls all makes and models comprising the equipment involved in the installation, managing all the tallies and the labeling) in a centralized fashion. With an unlimited number of simultaneous users, each of them has an interface (hardware or software panel) adapted to the technical and operational requirements of their role.
Regarding the control of the network, VSM gets integrated with the network electronics through the DCNM (Data Center Network Management) API, in such a way that it sends the orders that trigger the provisioning of new flows in the network, being the determination of the path of these flows left in the hands of IPFM’s NBM function. Subsequently, VSM receives confirmation and information on the characteristics and route of the established flows and presents it to the operator by means of tools that are more adapted to a broadcast environment.
In this sense, the monitoring tools offered by VSM for this project provide both a graphical representation of the network topology with all endpoints, their location and how they are interconnected (SmartDASH), and a real-time, subscription-enabled analysis of the streams (at media packet level) existing on the network (SmartSCOPE).
Cisco DCNM is also a valuable network monitoring and setup tool -more oriented to IT engineers- through which the network will be able to be configured and the host and flow policies that are decided can be applied in order to protect the network against flows from unauthorized devices or bandwidths that exceed those supported by the equipment, working in the format already defined for this project.
Last, and integrated with the tools described above, there is an alert system (vSNMP) that informs about the alarms that are triggered in the various devices by monitoring certain parameters (temperature, status of sources, etc.).
Keys to success
In our opinion and based on the experience gathered by now, the keys to this project’s success lie on the following pillars:
Pre-training: Receiving the appropriate training, from the hand of those who have the required expertise is the key to knowing the market and guiding both the drafting of the project and its execution on the right track.
Interdepartmental cooperation: Nobody knows a lot about everything. Therefore, turning to those who have a better knowledge of each system is a great support and provides assurance that no gaps are left unresolved. In addition, it can help shape up solutions to the problems in the most successful way. In the end, the whole company joins forces in pursuing the same goal, thus increasing the commitment of members while making them feel they are part of a whole.
Visit to facilities: Being able to see first-hand how similar things have been done with good results in other parts of the world provides assurance that we are at the right time to go for a change. These visits also provide ideas and market trends.
Awareness of market movements: Market trends are a good indication of where the strongest efforts are being made. The fact that one format is more popular than another is a telling sign of how it will survive the future, especially if there is also a clear adoption by the more established companies, which will also guarantee a more extensive implementation.
Commitment with standards: It is both prudent and essential to require partnering companies to commit to adopt in a foreseeable future the standards on which JT-NM is currently working. Although not everything is developed at the same moment in time, it must always be expected that what is on the approvals pipeline will be adopted by the company with which a relationship exists. Otherwise, we would move away from interoperability in a short time, thus accelerating the system obsolescence process.
Target the solution to a variety of manufacturers: Focusing the solution exclusively on a single manufacturer leads away from the important concept of interoperability. There is nothing more successful for showing a commitment towards interoperability than to demonstrate it through integration of different brands, choosing from each one the elements that best meet the relevant needs. In the case of network electronics -the basis for the transport layer of media flows- going for COTS switches has been a decisive step. They are the top experts 0n IP transport and they offer a wide range of models from which choosing the most suitable one for the project.
Ongoing communication with the ‘Client’: Know the needs of the ‘client’ to whom the project is carried out, open channels for involvement in the project so that they can contribute their experience, peculiarities and way of working, thus allowing to reach the most balanced solution possible between their company goals and policies. At the same time, it reduces the stress in the face of the change that is generated among the staff, as they feel taken into account and involved. The latter is the key to increasing their involvement and enthusiasm, which will in turn facilitate a change of mentality that they necessarily have to go through.
Ongoing training: Acquisition of knowledge generates confidence, and provides security by being aware that what we do has a theoretical foundation and has also proved to work well in other places. With good training, there are fewer and fewer loose ends and faith in the project increases. This training should be provided from the very beginning: prior to implementation it will raise awareness of what can be expected from the system. It will also facilitate a correct compromise between what is needed and how it can be achieved. After implementation, it will be useful for strengthening knowledge and processes and ways of taking action once the system is in operation.
Continuous monitoring: During the implementation process, nothing should be left unchecked or not made subject to approval. Systems are complex and we cannot afford to stray by not participating in all aspects of system setup, leaving everything the integrator’s discretion. Said criterion must be made clear and understood, be open to contributions or improvements; and finally, with the adoption or not of changes, approved and applied.