.. index:: installation, configuration, environment, Scolvo project, start, begin Getting started ============================================= To be able to work on a Scolvo Project the following steps should be done (the order is just a proposal, different sequence can be appropriate): - Install the Scolvo VS Code Extension with its dependencies - Acquire Scolvo access - Configure the Scolvo VS Code Extension - Create new Project/Environment - Save the developer license file - Plan your project - Prepare database structure of the components - Update existing script codebase to reflect your project domain .. index:: Install VS Code extension .. _getting_started_page_install_vs_code_extension: Install the Scolvo VS Code Extension with its dependencies ---------------------------------------------------------- It is recommended to have Docker Desktop installed and running while working with Scolvo Development Platform. The Docker Desktop can be downloaded and installed from `Docker Desktop official site `_. When working with the Scolvo Development Platform configure at least 4 GB memory resource to Docker to avoid container restart. The next step is the installation of Scolvo VS Code Extension: the easiest way to install it is to user the VS Code Extensions view and filter for *Scolvo* keyword. When the extension is displayed, then activate the *Install* button. After the installation a restart is required. .. image:: images/installScolvoExtension.png :width: 600 :alt: Install Scolvo Extension from Marketplace view :class: with-border Finally, it is recommended to install the Docker extension too, as it is a great help when dealing with Docker containers. .. index:: acquire Scolvo access Acquire Scolvo access --------------------- If you do not have a Scolvo access yet, then you can register your account on `Scolvo Playground `_. It is required to be able to Start your Environment (used when downloading Framework container images). .. image:: images/registerAccount.png :width: 600 :alt: Register your account :class: with-border After a successful registration you'll receive a welcome email containing your developer license. .. index:: configure VS Code Extension Configure the Scolvo VS Code Extension -------------------------------------- The Scolvo VS Code Extension requires configuration parameters to be set to be able to work correctly. These are: * workspace directory for Scolvo projects, * account username, * and account password. Open the *File/Preferences/Settings* window, select *Extensions/Scolvo* and set the above listed parameters. .. image:: images/extensionSettings.png :width: 600 :alt: Configure Scolvo Extension :class: with-border If you start an environment without configuration values, then the Extension shows input fiends on the fly (and stores them to the configuration). .. index:: create new Project/Environment Create new Project/Environment ------------------------------ If you have a Scolvo account, the Scolvo Extension installed, Docker Desktop up and running (and Docker extension installed), then it is time to create your first Scolvo Environment. Open *Scolvo view* and select *Create Environment* button. The extension now asks few parameters for new project creation (project name and default language). .. image:: images/createEnvironment.png :width: 600 :alt: Create new Environment :class: with-border When input parameters are defined for new project, then it downloads content from Scolvo remote services and opens the project. .. image:: images/openNewProject.png :width: 600 :alt: Created project :class: with-border Note: Actually "en", "de" and "hu" default languages are supported on Login screens, but this can be extended on demand. .. index:: save license file Save the developer license file ------------------------------- After a successful Scolvo account registration you own a developer license file what can be used during local development. This file has to be copied (or the content of this file) to the *license* folder of your environment with name *scolvo.license*. This allows you to start the Scolvo environment locally, with maximum 3 users created. .. image:: images/copyLicenseFile.png :width: 600 :alt: Created project :class: with-border If the license is expired then login to your `Scolvo Playground profile `_ and request a new one. Plan your project ------------------ It is recommended to define your project domain model before starting the feature implementation. Think about what data needed where, and what is the minimal data structure for clients (different model can be defined for Mobile and Admin use cases). .. index:: design database structure Prepare database structure of the components -------------------------------------------- When planning a solution in Scolvo Development Platform following 3 components require database design: - Backend, - Admin (mhub), - and Mobile (base for webview too) Tables required by the system are already defined in starter projects in form of Liquibase files. To separate them from the domain of the customer project the have a name *0.xml*. It is advised to not change thses, but add new tables or new columns of existing tables in separate files (1.xml, 2.xml, etc). The different files are bound in changeLog.xml files (this name is configured, do not change it), so if you add new change descriptor xml, then do not forget extending them. **Working with Backend data structure** The Backend's Database is persisted, it also stores the Liquibase version information, therefore sometimes it is necessary to reset your project to wipe out data (and data structure) from database and from authentication database too. (How to reset the project is described in separate section under Environment operation.). The only domain table in the starting content is the *user* table and its history table with name *user_history*. - If you extend the structure of the *user* table then extend the same time its history table with same columns (but do not use not-null constraints) - If you add a new table to the Backend data structure, then create immediately its history table (e.g. bird and bird_history). All backend domain table must have mandatory columns to be added (see the user table example: createdAt, createdBy, updatedAt, updatedBy). - Use type VARCHAR(36) for id columns (uuid() built-in function generates this type), make them PK. - Define indexes for possible filtering and for foreign keys (real FKs are not recommended to use, but the indexes are recommended) to not lose performance when the tables are populated. **Working with Admin data structure** The database of an Admin session is memory based, no need to restart/reset anything when the schema is changed, a logout/login is enough to be done. The relevant files are under the *mhub* directory. There is a *0.xml* defined to separate tables created for the system use (user table for user administration feature). There is no history feature in the client, so these tables are not defined in the database structure, and usually columns storing history related information are not added (createdAt, createdBy, updatedAt, updatedBy). - If you extend the structure of the *user* table for Backend database and this data is UI relevant, then extend accordingly the definition of the *user* table in the mhub folder by adding a new change definition file. - If you add a new table to the Admin data structure, then use same types as for the Backend definition of the same table: use VARCHAR(36) for id columns (uuid() built-in function generates this type), make it PK. - Define indexes for possible filtering and for foreign keys (FKs are not defined, but the indexes are required) to not lose performance when the tables are populated. - Transfer only the required data structure and data set to the client to keep database as lightweight and performant as possible. **Working with Mobile data structure** The database of a Mobile session is memory based, no need to restart/reset anything when the schema is changed, a logout/login is enough to be done. The relevant files are under the *mobile* directory. There is no *0.xml* file defined, as there is no predefined structure for mobile apps. According to this behavior new definitions are placed immediately to *1.xml* file. There is no history feature in the client, so these tables are not defined in the database structure, and usually columns storing history related information are not added (createdAt, createdBy, updatedAt, updatedBy). - If you add a new table to the Mobile data structure, then use same types as for the Backend definition of the same table: use VARCHAR(36) for id columns (uuid() built-in function generates this type), make it PK. - Define indexes for possible filtering and for foreign keys (FKs are not defined, but the indexes are required) to not lose performance when the tables are populated. - Transfer only required data structure and data set to the client to keep database as lightweight and performant as possible. **Best practice on data types** - Define **id** columns as **VARCHAR(36)** and you can define unique value using *uuid()* built-in function. - Define **date-time** and **date** values as **BIGINT**, there are many useful conversion methods defined, check details in :ref:`Date functions`. - You must define **boolean** types as **VARCHAR(8)**, this type is specially serialized/deserialized by the Framework. When you deal with this value in scripts, then *normal* boolean values (**true**, **false**) can be used. - It is a good habit to use reference id-s to other tables, but usually we do not use FKs. The referring column has a name *Id* (e.g. birdId for referring a *bird* entity). As it is a FK-like column, an index has to be created for this column. .. index:: update scripts for database changes Update existing script codebase to reflect your project domain -------------------------------------------------------------- The starting script content of your Scolvo environment reflects to the starting data structure, if you for example extend the structure with a new table, then the script set has to be extended accordingly. In our example we define a simple new table **bird** having the following structure: **Backend, bird table definition** - id - VARCHAR(36), PK - name - VARCHAR(50), Not null - birthDate - BIGINT, Not null - description - VARCHAR(200) - createdAt - BIGINT, Not null - createdBy - VARCHAR(36), Not null - updatedAt - BIGINT - updatedBy - VARCHAR(36) - indexes: - *id* has index, as defined as PK - *createdBy* should have and index, as it is a userId FK - *updatedBy* should have and index, as it is a userId FK **Backend, bird_history table definition** - id - VARCHAR(36) - name - VARCHAR(50) - birthDate - BIGINT - description - VARCHAR(200) - createdAt - BIGINT - createdBy - VARCHAR(36) - updatedAt - BIGINT - updatedBy - VARCHAR(36) - indexes: - no indexes are defined, even PK must not be defined **Mobile, bird table definition** - id - VARCHAR(36), PK - name - VARCHAR(50), Not null - birthDate - BIGINT, Not null - description - VARCHAR(200) - index definitions: - *id* has index, as defined as PK - *name* should have and index, for filtering - *birthDate* should have an index, as in our example a range filtering is planned **Backend script change, /backend/TypeDefinitionLifeCycle.scolvo** This script contains database change events, where a strict name pattern is defined for the db operations. You have to create below functions for our example, in case INSERT, UPDATE and DELETE operations are all supported: - onBirdInserting(originId) - onBirdInserted(originId) - onBirdUpdating(originId) - onBirdUpdated(originId) - onBirdDeleting(originId) - onBirdDeleted(originId) According to the logic of the domain, they can hold code for *pre-* and *post-processing* of DB operation (or can be empty if there is no such logic). The row representation can be accesses through *$IN.data.map* context reference (e.g. var birdDao = $IN.data.map;). **Backend script change, /backend/repository/BirdRepository.scolvo** The repository script is usually a new script for a new entity, it holds the different database *select* definitions according to the needs. Normally a function is created for data synchronization collecting all relevant rows in appropriate structure (matching the client data structure). For our example we create the BirdRepository.scolvo file containing only one function collecting all birds with all client side defined columns: .. code-block:: scolvoscript :caption: BirdRepository.scolvo example :linenos: function getBirds() { return select() .selectAs("id") .selectAs("name") .selectAs("birthDate") .selectAs("descripton") .from("bird") .execute(); } **Backend script change, /backend/DataSync.scolvo** The DataSync scolvo file contains the event definition of onDataSync(userId, deviceType), what is called, when a successful login is processed in one of the clients. In this case the Backend is requested to collect all relevant data (as minimal as possible) to be sent to the client database. Here you can make decisions based on the user role and the client type. In our case we can differentiate between pre-defined role "superUser", a usually defined role "administrator" and the domain specific "zooKeeper" role. Do not forget to import new scolvo file before using its functions! Here is the extended function: .. code-block:: scolvoscript :caption: DataSync.scolvo handling zooKeepers :linenos: import { /common/Roles, /backend/repository/UserRepository, /backend/repository/BirdRepository } function onDataSync(userId, deviceType) { info("onDataSync for " + userId + ", deviceType: '" + deviceType + "'"); if (userId == "superUser") { if (deviceType == "MHUB") { return dataSyncByAdministrator(); } return []; } var user = getUserById(userId); if (user.role == roleAdministrator && deviceType == "MHUB") { return dataSyncByAdministrator(); } if (user.role == "zooKeeper") { return dataSyncByZooKeeper(); } warn("DataSync returs empty for role " + user.role); return []; } function dataSyncByAdministrator() { return [ createDataSyncElement("user", getUsers()), ]; } function dataSyncByZooKeeper() { return [ createDataSyncElement("bird", getBirds()), ]; } **Client side script change, /mobile/DataChange.scolvo** This script contains the event invoked when a data change arrives from Backend. Here you can decide what to do with the received information, how the client has to react to that. Usually we check the table name and the operation, and based on that, we can execute or skip data handling. This is the place where you can include the logic for reactive behavior. For our example, we simply save the received *bird* table changes, and refresh the lists on the UI, if needed (we expect that Birds page is already defined with relevant refresh function *refreshBirdList*): .. code-block:: scolvoscript :caption: DataChange.scolvo handling *bird* changes :linenos: import { /mobile/SessionUser, /mobile/bird/Birds } function onDataChangeDg(originId) { debug("Data change message arrived ..."); if (getCurrentUser() == null) { return null; } var changeset = $IN.changeset; var typeDefinition = $IN.typeDefinition; debug("Data change message arrive with type definition: " + typeDefinition + ", size is: " + changeset.size()); if (typeDefinition == "bird") { changeset.each(function(dao) { if (dao.changeType == "INSERT") { insertOrReplaceTypeDefinition(typeDefinition, dao); } else if (dao.changeType == "UPDATE") { updateTypeDefinition(typeDefinition, dao.id, dao); } else { deleteTypeDefinition(typeDefinition, dao.id); } }); refreshBirdList(originId); } } Environment operation ~~~~~~~~~~~~~~~~~~~~~ Different phases of the development requires different actions to be executed in the environment. Consider following hints: - Running state only required when you want to use admin view or mobile view. - Changes (database schema, script) in admin (mhub) and mobile scripts require only a logout-login process. - Changes in the Backend database schema or scripts require Backend restart, or environment restart. - Usage of admin and mobile views require created user and logged-in in the environment. You can use Admin view for user creation. Following cases are described in detail below: - Start, stop and restart - Backend restart - Login to admin view - Login to web view to debug mobile scripts - Creating new user - Resetting the environment .. index:: environment operation: start .. index:: environment operation: stop .. index:: environment operation: restart Start, stop and restart ----------------------- .. note:: Be aware that the environment uses different ports operate, what have to be free when starting an environment. Following ports are used by a local environment: - **80**, **443** for *proxy* component, - **5671**, **5672**, **15672** for *mq* component, - **13306** for *authdb* component, - **23306** for *bedb* component Make sure that these ports are free, and the Docker Desktop is running before the VS Code is started. The Scolvo VS Code Extension supports all necessary operations to be able to start, stop or restart the environment. You can use the *Run and Debug* view to do that: - The extension checks for running images, and if the same environment is running, then it attaches the log and controlling panel. - If the environment is not started yet, you can use *Start Debugging* button of *Run and Debug* view to start the environment. Relevant log lines are listed in *Output* view at first, and later in the *Debug Console*. The console shows log lines for *Backend*, *Admin* and *Webapp* (compatible with a mobile client) components. - Using the floating control panel of the *Run and Debug* view the environment can be stopped/started or restarted any time. - If the *Run and Debug* view is not in synch with the running state of the environment images, then the *Docker* Extension view can be used to stop the environment and get a synchronized state again (so you are able to use the *Run and Debug* view correctly again). .. image:: images/startStopRestart.png :width: 600 :alt: Start, stop and restart the environment :class: with-border .. index:: environment operation: Backend restart Backend restart --------------- Changes in the Backend database schema or scripts require Backend restart (restart of the whole environment is suitable too). A dedicated quick link with name *Restart backend* can be found in the *Scolvo* view's *Help and Feedback* section performing the required action. There is no need to logout/login to the existing sessions, when the Backend is running again, then the usage can be continued smoothly. (If the change is in the data synchronization part, then a logout/login is required anyway.) .. image:: images/restartBackend.png :width: 600 :alt: Backend restart :class: with-border .. index:: environment operation: login to Admin view Login to Admin view -------------------- When testing changes in the Admin part (under mhub folder) of the scripts, it is required to login to an Admin session. A dedicated quick link with name *Open Admin* can be found in the *Scolvo* view's *Help and Feedback* section performing the required action. .. image:: images/loginToAdmin.png :width: 600 :alt: Login to Admin :class: with-border This opens a new tab with the login screen in the default browser of your OS (Chrome and Firefox browsers are supported). An existing user of the local environment (created beforehand) is required. After successful login, the menu structure and default page is displayed according to the script content. .. image:: images/loginToAdminExternalBrowser.png :width: 600 :alt: Login screen of Admin :class: with-border .. index:: environment operation: login to Web view (mobile view) Login to web view to debug mobile scripts ------------------------------------------ As the mobile apps are too complex to use them locally, the web view represents their functionality while implementing the project. When testing changes in the Mobile part (under mobile folder) of the scripts, it is required to login to a WebApp session. A dedicated quick link with name *Open WebApp* can be found in the *Scolvo* view's *Help and Feedback* section performing the required action. .. image:: images/loginToWebapp.png :width: 600 :alt: Login to WebApp :class: with-border This opens a new tab with the login screen in the default browser of your OS (Chrome and Firefox browsers are supported). An existing user of the local environment (created beforehand) is required. After successful login, the menu structure and default page is displayed according to the script content. .. image:: images/loginToAdminExternalBrowser.png :width: 600 :alt: Login screen of WebApp :class: with-border .. index:: environment operation: create new user Creating new user ------------------ Local user accounts are required to be able to login to Admin and WebApp sessions. A new user can be registered using Admin scripts and default super administrator user credentials (username/password is *su*/*su*). After successful login the default page is the Users page, where the existing users of the environment are listed. Here you can create, modify, disable and enable users. The relevant scripts are under the mhub/users folder and in the common/Roles.scolvo script where the logic and roles in your environment can be updated according to your needs. .. image:: images/userManagement.png :width: 600 :alt: Managing users :class: with-border .. index:: environment operation: reset Resetting the environment -------------------------- The state of the local environment might become unusable due to radical database structure or logical change needing to wipe out the whole environment and start with a fresh new. For these cases we can use the Reset environment operation. This can be invoked by `opening the VS Code command palette `_ and filter for Scolvo keyword and finally activate *Reset scolvo project* command. It is advised to stop the environment before resetting it. .. image:: images/resetProject.png :width: 600 :alt: Managing users :class: with-border This command removes the container instances, so after starting the environment, a brand-new environment is set up.