Currently, in many data landscapes, the information is distributed across various sources and presented in diverse formats. This fragmentation can pose a significant challenge to the efficient application of analytical methods. In this sense, distributed data mining is mainly based on clustering or classification techniques, which are easier to implement in distributed environments. However, the solution to some problems is based on the usage of mathematical equations or stochastic models, which are more difficult to implement in distributed environments. Usually, these types of problems need to centralize the required information, and then a modelling technique is applied. In some environments, this centralization may cause an overloading of the communication channels due to massive data transmission and may also cause privacy issues when sending sensitive data. To mitigate this problem, this paper describes a general-purpose distributed analytic platform based on edge computing for distributed networks. Through the distributed analytical engine (DAE), the calculation process of the expressions (that requires data from diverse sources) is decomposed and distributed between the existing nodes, and this allows sending partial results without exchanging the original information. In this way, the master node ultimately obtains the result of the expressions. The proposed solution is examined using three different computational intelligence algorithms, i.e., genetic algorithm, genetic algorithm with evolution control, and particle swarm optimization, to decompose the expression to be calculated and to distribute the calculation tasks between the existing nodes. This engine has been successfully applied in a case study focused on the calculation of key performance indicators of a smart grid, achieving a reduction in the number of communication messages by more than 91% compared to the traditional approach.
Keywords: data integration; edge computing; genetic algorithm; particle swarm optimization; smart grid.