.. index:: Scolvo script, Data Cycle **************** Data Cycle **************** This page describes in detail how the client produced data is distributed in the Scolvo Development Platform. The data change distribution is an essential topic in Scolvo Development Platform as it makes possible to: - Spread the project specific information changes among the central components and connected clients, - Be reactive on the client UIs, if the business use-case request it, - Give prompt information about the changes to the connected external systems. There are two main cases for data changes described here in details: - Client data change flow, - And Backend data change flow. Client data change flow ========================= In this flow a user of a connected client make changes in the local database and sends this information towards the Backend component too. The Backend receives the Data Change Request and triggers the appropriate events before and after executing the changes in the Backend database, and finally distributes the changes to all connected clients (including the originating one). The connected clients finally receive the description of the changes and react according to the implemented business logic (they can accept and execute or ignore the changes). Below you can find code examples for each step with hints and descriptions. Step 1: A client makes changes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is a usual case, that based on a UI action (e.g. push the Save button) the client creates a new data to be inserted locally, and it sends the change to Backend at the same time. In our example when the user activates *AddClientDataChange* button, then the SVM invokes the *onAddClientDataChange()* event, where the data changes are implemented. In this example case we use the *note* table defined in Playground and use it for our actual purpose extending it with unique data. .. code-block:: scolvoscript :caption: The client creates new data and handles it on ClientDataChangePage page :linenos: import { /mobile/repository/NoteRepository } function displayClientDataChangePage(originId) { debug("Display ClientDataChange Page ..."); var dataRows = clientDataChangeGetData(); var pageData = { "ClientDataChangePage": { "ClientDataChangeList": dataRows } }; display(ClientDataChangePage, pageData, originId); } function clientDataChangeGetData() { return getNotes().map(function (noteDao) { return { "id": noteDao.id, "title": noteDao.title, "content": noteDao.content }; }); } function clientDataChangeRefreshList(originId) { fireEvent(createRefreshItemTargetEvent("ClientDataChangeList", getNotesData()), "ClientDataChangePage"); } function clientDataChangeRemoveIdFromList(recordId, originId) { fireEvent(createRemoveItemTargetEvent("ClientDataChangeList", recordId), originId); } function onClientDataChangePageLoaded(originId) {} page ClientDataChangePage { layout: vertical; template: general; settingsVisible: true; scolvoMenuVisible: true; list ClientDataChangeList { template: listVerticalNormal; filterVisible: true; span: 0; paged: false; itemTemplate: listItemMultiLine; actions: [ AddClientDataChange ] columns: [ mainText => title, subText => content ] } } function onClientDataChangeListFilterCount(originId) { return clientDataChangeGetData().size(); } function onClientDataChangeListSelectionChanged(originId) { debug("On list selection changed: " + $IN.data); // Do nothing actually } function onAddClientDataChange(originId) { var newTitle = dateToString(nowInMillis(), "yyyy.MM.dd. HH:mm"); var newContent = "The content of " + newTitle; var dataDao = { "id": uuid(), "title": newTitle, "content": newContent, "changeType": "INSERT" }; insertTypeDefinition("note", dataDao); sendDataChange("note", [dataDao]); fireEvent(createAddItemTargetEvent("ClientDataChangeList", dataDao), "ClientDataChangePage"); } .. The following main actions are done in the *onAddClientDataChange()* event implementation: - A *DAO* object is created with a unique *id* and details, - The local database is changed by the *insertTypeDefinition()* function call, - The sendDataChange() function call sends the change detils to the Backend for processing, - The *fireEvent* call with the created *AddItemTargetEvent* object refreshes the UI and populates a new item in the list. Step 2: The Backend receives the change ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When the Backend receives a Data Change Request, then for each row it executes a *pre*, and *post* change event with appropriate name. The event names are calculated based on the *changeType* and on the *type definition name* (table name). As in our example the *note* table is used, the following events are in focus: - **onNoteInserting(originId)** - event executed before changing the Backend database (pre INSERT script), it can block the insert if failed - **onNoteInserted(originId)** - event executed after changing the Backend database (post INSERT script), executed asynchronously, no effect on database operation - **onNoteUpdating(originId)** -event executed before changing the Backend database (pre UPDATE script), it can block the update if failed - **onNoteUpdated(originId)** - event executed after changing the Backend database (post UPDATE script), executed asynchronously, no effect on database operation - **onNoteDeleting(originId)** - event executed before changing the Backend database (pre DELETE script), it can block the delete if failed - **onNoteDeleted(originId)** - event executed after changing the Backend database (post DELETE script), executed asynchronously, no effect on database operation In these event hooks it is possible to implement customer specific logic according to the requirements (e.g. sending information about the change to an external system). In the actual example an email is sent to the creator user with the new record details. This kind of event definitions are usually placed to *TypeDefinitionLifeCycle.scolvo* under the *backend* script directory. The context of the event has the following information (the context can be accessed through *$IN.data* map): - **clientId** - the unique ID of the client, - **createdBy** - the ID of the creator user in User table, - **messageCreated** - the time stamp of the Data Change message, - **language** - the used locale of the client (browser or mobile client), - **map** - the DAO object of the change in a Map The creator user deatils are retrieved by the *createdBy* ID containing name and email for the *sendEmail()* built-in function call. .. code-block:: scolvoscript :caption: Backend data change events for *note* table in *TypeDefinitionLifeCycle.scolvo* :linenos: // Note related data change event definitions: function onNoteInserting(originId) {} function onNoteInserted(originId) { var noteDao = $IN.data.map; debug("The request information is: " + $IN.data); var creatorUser = getUserById($IN.data.createdBy); if (creatorUser != null) { debug("The creator user's email is: " + creatorUser.email); sendEmail( getEnv("EMAIL_USERNAME"), creatorUser.email, "", "New Note data created", "You have created a new data: " + noteDao + ". Have a nice day!", null); debug("The email has been sent successfully"); } } function onNoteUpdating(originId) {} function onNoteUpdated(originId) {} function onNoteDeleting(originId) {} function onNoteDeleted(originId) {} .. Step 3: The Backend distributes the change to the clients ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When the data change is executed in the central database the Backend sends a *Data Change Datagram* message to the clients. The message sending is usually parallel to the *post* event execution in the Backend component. Step 4: An online client receives the change ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Every online client gets the *Data Change Datagram* message, and a specific event with name *onDataChangeDg(originId)* executed to process it. This event is usually placed to DataChange.scolvo file. The implementation of the event has to reflect the business requirements. In our case the mobile script set (mobile apps and the Webapp client) logic is to handle *note* table changes and execute them on local database. This event is the place to handle the reactive behavior of the system if required. In this case it is implemented to refresh the list on our page. Note: The built-in function *insertOrReplaceTypeDefinition()* called to execute INSERT database operation as the originating client receives the changes too and the *note* table already contains the data. .. code-block:: scolvoscript :caption: The *onDataChangeDg(originId)* event imlpementation in mobile/DataChange.scolvo script file :linenos: import { /mobile/SessionUser, /mobile/repository/NoteRepository, /mobile/example/ClientDataChangePage } function onDataChangeDg(originId) { debug("Data change message arrived ..."); if (getCurrentUser() == null) { return null; } var changeset = $IN.changeset; var typeDefinition = $IN.typeDefinition; debug("Data change message arrive with type definition: " + typeDefinition + ", size: " + changeset.size()); if (typeDefinition == "note") { changeset.each(function(dao) { if (dao.changeType == "INSERT") { insertOrReplaceTypeDefinition(typeDefinition, dao); } else if (dao.changeType == "UPDATE") { updateTypeDefinition(typeDefinition, dao.id, dao); } else { deleteTypeDefinition(typeDefinition, dao.id); } }); clientDataChangeRefreshList(originId); } } .. The event context has the *DataChangeDg* object, what can be accessed through the context by the **$IN** expression. The object has following accessible keys: - **typeDefintion** - the name of the database type, - **typeDefinitionVersion** - the version of the change, time in milliseconds of UTC time-zone, - **changeset** - collection of data rows containing the *changeType* describing the desired database operation (INSERT, UPDATE, DELETE). When updating or deleting data by the clients ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ When the business logic requires update or deletion of a business object, the example code above has to be modified. For the update operation the *changeType* is **UPDATE** for the delete operation the appropriate *changeType* is the **DELETE**. The built-in function for refreshing the list of the entities is different too: for the update case the *createReplaceItemTargetEvent()*, for the delete case the *createRemoveItemTargetEvent()* built-in function has to be used. **Changes for the update and delete cases** A set of row actions is introduced on the list data to be able to do specific operation on the selected row, as these (e.g. the update and delete) operations require the unique Id of the data. To be able to show these row actions the layout of the list is switched to *listItemCard*, and a new set of row operations and clicked events are introduced. .. code-block:: scolvoscript :caption: Changes for Update and Delete operations in the ClientDataChangePage.scolvo :linenos: import { /mobile/repository/NoteRepository } function displayClientDataChangePage(originId) { debug("Display ClientDataChange Page ..."); var dataRows = clientDataChangeGetData(); var pageData = { "ClientDataChangePage": { "ClientDataChangeList": dataRows } }; display(ClientDataChangePage, pageData, originId); } function clientDataChangeGetData() { return getNotes().map(function (noteDao) { return buildClientDataChangeListRowFromDao(noteDao); }); } function clientDataChangeRefreshList(originId) { fireEvent(createRefreshItemTargetEvent("ClientDataChangeList", getNotesData()), "ClientDataChangePage"); } function clientDataChangeRemoveIdFromList(recordId, originId) { fireEvent(createRemoveItemTargetEvent("ClientDataChangeList", recordId), originId); } function onClientDataChangePageLoaded(originId) {} page ClientDataChangePage { layout: vertical; template: general; settingsVisible: true; scolvoMenuVisible: true; list ClientDataChangeList { template: listVerticalNormal; itemTemplate: listItemCard; filterVisible: true; span: 0; paged: false; actions: [ AddClientDataChange ] columns: [ title => title, text => content ] } } function onClientDataChangeListFilterCount(originId) { return clientDataChangeGetData().size(); } function onClientDataChangeListSelectionChanged(originId) { debug("On list selection changed: " + $IN.data); // Do nothing actually } function onAddClientDataChange(originId) { var newTitle = dateToString(nowInMillis(), "yyyy.MM.dd. HH:mm"); var newContent = "The content of " + newTitle; var dataDao = { "id": uuid(), "title": newTitle, "content": newContent, "changeType": "INSERT" }; insertTypeDefinition("note", dataDao); sendDataChange("note", [dataDao]); fireEvent(createAddItemTargetEvent("ClientDataChangeList", buildClientDataChangeListRowFromDao(dataDao)), "ClientDataChangePage"); } function buildClientDataChangeListRowFromDao(dataDao) { return { "id": dataDao.id, "title": dataDao.title, "content": dataDao.content, "actions": ["ModifyClientData", "DeleteClientData"] }; } function onModifyClientData(originId) { debug("Modify data id is: " + $IN.data.id); var newTitle = $IN.data.title + " ."; var newContent = $IN.data.content + " ."; var dataDao = { "id": $IN.data.id, "title": newTitle, "content": newContent, "changeType": "UPDATE" }; updateTypeDefinition("note", dataDao.id, dataDao); sendDataChange("note", [dataDao]); fireEvent(createReplaceItemTargetEvent("ClientDataChangeList", buildClientDataChangeListRowFromDao(dataDao)), "ClientDataChangePage"); } function onDeleteClientData(originId) { debug("Delete data: is to be deleted is:" + $IN.data.id); var dataDao = { "id": $IN.data.id, "changeType": "DELETE" }; deleteTypeDefinition("note", dataDao.id); sendDataChange("note", [dataDao]); fireEvent(createRemoveItemTargetEvent("ClientDataChangeList", dataDao.id), "ClientDataChangePage"); } .. Backend data change flow ========================= This data flow is used when a scheduled backend logic, data import or external system trigger results in data change to be distributed among clients. In the below example a periodic Backend trigger produces some data, and a mobile client reacts to that. Repeated jobs in Backend ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ The following example defines a periodic data creation for above defined data structure. .. code-block:: scolvoscript :caption: Job definition and Backend side produced data distribution to clients in Jobs.scolvo script :linenos: import { /backend/Broadcast } function initScheduledJobs() { addJob("0 */5 * ? * * *", "DataCreationJob", "ProduceData", "CreateData"); } //========================== Execute initScheduledJobs initScheduledJobs(); function onCreateData(originId) { debug("CreateData scheduled job - START"); var newTitle = "From Backend - " + dateToString(nowInMillis(), "yyyy.MM.dd. HH:mm"); var newContent = "The content of " + newTitle; var dataDao = { "id": uuid(), "title": newTitle, "content": newContent, "changeType": "INSERT" }; insertTypeDefinition("note", dataDao); publishBroadcast("note", [dataDao]); debug("CreateData scheduled job - END"); } .. As result the Backend produces new data in every 5 minutes, and if there is a connected client with opened Note listing page, then the client refreshes the list when a new data arrives.