Leszek Rutkowski - Selected Publications#

1) Jaroslaw Bilski, Leszek Rutkowski, Jacek Smolag, Dacheng Tao: A novel method for speed training acceleration of recurrent neural networks, Information Sciences 553: 266-279, 2021, https://doi.org/10.1016/j.ins.2020.10.025

SIGNIFICANCE: In this paper, a new and original idea of the Jordan recursive neural network is proposed and studied in detail. The basic assumption was to considerably decrease the time needed in both the recall/learning phases by using the parallel structures. This is very important for all neural networks, especially recurrent networks. The obtained architecture facilitates a very high speed of numerical operations. This structure implements both phases of the investigated RNN. The presented approach is especially significant in reference to autonomous, self-learning applications. The efficiency of the designed novel architecture is very impressive. Therefore, it is very beneficial for parallel implementations of big networks. The presented solutions can be implemented in digital hardware. The paper was published in 2021 in Information Sciences- a very prestigious journal in the area of computer science with an impact factor of 6.795, a so-far gained 7 citations.

2) Xiaoxiao Lv, Jinde Cao, Leszek Rutkowski, Dynamical and static multisynchronization analysis for coupled multistable memristive neural networks with hybrid control. Neural Networks 143, pp. 515-524,2021, DOI 10.1016/j.neunet.2021.07.004

SIGNIFICANCE: This paper is the first study in the world literature devoted to dynamical properties, namely the dynamical and static multisynchronization, of delayed coupled multistable memristive neural networks. By replacing the resistor in conventional neural networks with a memristor, the memristive neural networks can more precisely replicate the artificial neural networks of the human brain. They can be widely applied to a large variety of areas such as signal processing, face detection, bioinspired engineering. The multisynchronization problems have been solved by employing a new Halanay-type inequality and the impulsive control theory. This paper was published in 2021 in Neural Networks -a flagship journal of the European Neural Network Society and the International Neural Network Society, with an impact factor of 8.050.

3) Leszek Rutkowski, Maciej Jaworski, Piotr Duda, Stream Data Mining: Algorithms and Their Probabilistic Properties, Springer, 2020.
https://link.springer.com/book/10.1007/978-3-030-13962-9?noAccess=true#authorsandaffiliationsbook

SIGNIFICANCE: This book summarizes the original and breakthrough results of the research team led by Professor Leszek Rutkowski. This team has developed new methods and algorithms for stream data mining, presented in this book, and earlier in a series of papers (not listed in this section of the nomination form) published by the team in the most prestigious journals and attracted many citations: a) Rutkowski, L., Pietruczuk, L., Duda, P., Jaworski, M.: Decision trees for mining data streams based on the McDiarmid's bound. IEEE Trans. Knowl. Data Eng. 25(6) , 1272–1279 (2013), 211 citations; b) Rutkowski, L., Jaworski, M, Pietruczuk, L., Duda, P.: Decision Trees for Mining Data Streams Based on the Gaussian Approximation, IEEE Trans. on Knowledge and Data Engineering, vol. 26, no. 1, pp. 108-119, Jan. 2014, 165 citations; c) Jaworski, M., Duda, P., Rutkowski, L.: New splitting criteria for decision trees in stationary data streams. IEEE Trans. Neural Netw. Learn. Syst. 29, 2516–2529 (2018), 81 citations ; d) Rutkowski, L., Jaworski, M., Pietruczuk, L., Duda, P.: A new method for data stream mining based on the misclassification error. IEEE Trans. Neural Netw. Learn. Syst. 26(5), 1048–1059 (2015), 114 citations ; e) Pietruczuk, L., Rutkowski, L., Jaworski, M., Duda, P.: How to adjust an ensemble size in stream data mining? Inf. Sci. 381, 46–54 (2017), 66 citations.

In the above papers and in the above book, which summarizes the results of these papers, the authors have challenged the concept of the so-called Hoeffding Trees, proposed various new methods and algorithms, and derived formulas specifying a minimal number of required data in streaming data classification problems. The book presents a pioneering and unique approach to data stream mining problems, putting emphasis on the theoretical backgrounds of considered algorithms and focuses on algorithms that are mathematically justified. The book has been published in a very prestigious series, "Big Data" of the Springer publisher, and attracted 30 citations.

4) X. Yang, Y. Liu, J. Cao and L. Rutkowski, "Synchronization of Coupled Time-Delay Neural Networks With Mode-Dependent Average Dwell Time Switching," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 12, pp. 5483-5496, Dec. 2020, doi: 10.1109/TNNLS.2020.2968342.

SIGNIFICANCE: This paper presents a new concept concerning the synchronization of dynamic coupled systems. Such systems usually consist of a set of subsystems called nodes and interconnections among them called edges. A node can represent any system, such as the Lorenz system, Chua's circuit, Lure system, and neural network. The topology of a network, which can be directed or undirected, is usually described by an asymmetric or symmetric Laplacian matrix, respectively. Time delay is one of the most crucial factors when studying the dynamics of neural networks. This paper considers the model of coupled NNs with mixed delay, switching topology, and stochastic perturbations. For the first time in the literature, global exponential synchronization almost surely has been studied and formally proved. It should be emphasized that the presented method can be easily extended to other linear or nonlinear time-delay systems. This paper has been published in IEEE Transactions on Neural Networks and Learning Systems (impact factor – 10.451) – a flagship journal of the IEEE Computational Intelligence Society, and attracted, during one year, 31 citations.

5) P. Duda, L. Rutkowski, M. Jaworski and D. Rutkowska, "On the Parzen Kernel-Based Probability Density Function Learning Procedures Over Time-Varying Streaming Data With Applications to Pattern Classification," in IEEE Transactions on Cybernetics, vol. 50, no. 4, pp. 1683-1696, April 2020, doi: 10.1109/TCYB.2018.2877611.

SIGNIFICANCE: In this paper, for the first time in the literature, a recursive variant of the Parzen kernel density estimator has been proposed to track changes of dynamic density over data streams in a nonstationary environment. In stationary environments, well-established traditional techniques have nice asymptotic properties. Their existing extensions to deal with stream data are mostly based on various heuristic concepts, thus losing convergence properties. In this paper, a Parzen kernel-based recursive procedure, tracking concept drift, has been studied and its weak (in probability) and strong (with probability one) convergence have been proved, resulting in perfect tracking properties as the sample size approaches infinity. In three theorems, it is shown how to choose the bandwidth and learning rate of a recursive procedure in order to ensure weak and strong convergence. The results are applicable to both density estimation and classification over time-varying stream data. The paper was published in 2020 by IEEE Transactions on Cybernetics - a very prestigious journal with an impact factor of 11.448. This is the flagship journal of IEEE Systems, Man and Cybernetics Society. This paper has already gained 33 citations.

6) Y. Liu, Y. Zheng, J. Lu, J. Cao and L.Rutkowski, "Constrained Quaternion-Variable Convex Optimization: A Quaternion-Valued Recurrent Neural Network Approach," in IEEE Transactions on Neural Networks and Learning Systems, vol. 31, no. 3, pp. 1022-1035, March 2020, doi: 10.1109/TNNLS.2019.2916597.

SIGNIFICANCE: This paper proposes a breakthrough quaternion-various recurrent neural network (QVNN) approach, which is aimed at solving a large class of nonsmooth programming for the quaternion-valued optimization problems. This paper is motivated by many practical applications in the quaternion field, such as face recognition, video compression, and in-flight alignment optimization for airborne position and orientation systems. It is demonstrated that the state of the designed neural network arrives at the feasible region in finite time and stays there thereafter. More precisely, via chain rules and Lyapunov theorem, it is shown that a quaternion-various recurrent neural network stabilizes the system dynamics and converges to the optimal solution of the studied constrained convex optimization problem. Compared with the existing related quaternion neural network, the presented neural network can cope with the convex optimization directly in the quaternion field without decomposing. This is the first result in the world scientific literature and attracted very many citations. This paper has been published in IEEE Transactions on Neural Networks and Learning Systems (impact factor – 10.451) – a flagship journal of the IEEE Computational Intelligence Society, and attracted, during one year, 59 citations.

7 )T. Yu, J. Cao, L. Rutkowski and Y. -P. Luo, "Finite-Time Synchronization of Complex-Valued Memristive-Based Neural Networks Via Hybrid Control," in IEEE Transactions on Neural Networks and Learning Systems, doi: 10.1109/TNNLS.2021.3054967. (Early Access – available online).

SIGNIFICANCE: This paper, for the first time in the literature, shows finite-time stability and synchronization properties of complex-valued neural networks (CVNN). In many applications, the memristive-based neural networks (MNN) can be used to simulate the biological synapse in which the system with a memristor shows higher accuracy because of the properties of memory and nanoscale properties. Therefore, the stability problem of MNN has attracted much attention and is a hot topic of research. Different from the neural networks in the real field, the parameters of complex valued neural networks (CVNN) are defined in the complex field and various CVNNs can be used to solve some issues

which cannot be settled by a real-valued neuron. As a consequence, it is necessary to investigate the dynamical behaviors, such as stability, synchronization, bifurcation, and chaos of CVNNs. Different from the exponential/asymptotical stability, finite-time stability (FTS) requires that the trajectory can reach the attractor in a finite time rather than infinite time. The most interesting result – the finite-time stability is presented by making use of a novel Lyapunov-based finite-time convergency criterion based on impulsive differential inequality.

This paper has been recently published (Early Access) in IEEE Transactions on Neural Networks and Learning Systems (impact factor – 10.451) – a flagship journal of the IEEE Computational Intelligence Society and during few months attracted 6 citations.

8) 2) J. Wang, C. Yang, H. Shen, J. Cao and L. Rutkowski, "Sliding-Mode Control for Slow-Sampling Singularly Perturbed Systems Subject to Markov Jump Parameters," in IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51, no. 12, pp. 7579-7586, Dec. 2021, doi: 10.1109/TSMC.2020.2979860.

SIGNIFICANCE: This article presents a new approach to the investigation of sliding-mode control (SMC) for slow-sampling singularly perturbed systems (SPSs) with Markov jump parameters. In order to design a sliding-mode controller to ensure the stability of the proposed system, a novel integral sliding surface is developed. From the theoretical point of view, the most important is that we derived sufficient conditions to ensure that the state trajectories of the system are driven to a predefined sliding surface. Moreover, it is proved that the closed-loop sliding mode dynamics are stochastically stable. The paper was published in 2021 in the IEEE Transactions on Systems, Man and Cybernetics: Systems - a very prestigious journal with an impact factor of 13.451. This is the flagship journal of IEEE Systems, Man and Cybernetics Society. This paper has already gained 79 citations.

9) P. Duda, M. Jaworski, L. Rutkowski (2018), Convergent Time-varying Regression Models for Data Streams: Tracking Concept Drift by the Recursive Parzen-based Generalized Regression Neural Networks, International Journal of Neural Systems, vol. 28, No. 02, March 2018, https://doi.org/10.1142/S0129065717500484

SIGNIFICANCE: In this paper, for the first time in the world literature mathematically justified regression models, working in a time-varying environment, have been proposed and investigated. More specifically, the incremental versions of generalized regression neural networks are studied in the context of stream data mining. Their tracking properties — weak (in probability) and strong (with probability one) convergence is established assuming various concept drift scenarios. Several types of concept drifts are handled by the proposed approach in such a way that weak and strong convergence holds under certain conditions easily verified. The paper was published in 2018 in International Journal of Neural Systems - a very prestigious journal in the area of neural networks with the current impact factor of 5.866 and attracted 30 citations.

10) Korytkowski M., Rutkowski L., Scherer R., Fast image classification by boosting fuzzy classifiers (2016), Information Sciences, vol. 327, pp. 175 – 182. DOI https://doi.org/10.1016/j.ins.2015.08.030

SIGNIFICANCE: This paper presents a novel approach to visual objects classification based on generating simple fuzzy classifiers using local image features to distinguish between one known class and other classes. Boosting meta-learning is used to find the most representative local features. The proposed approach is tested on a state-of-the-art image dataset and compared with the bag-of-features image representation model combined with the Support Vector Machine classification. The novel method gives better classification accuracy and the time of learning and testing process is more than 30% shorter. It demonstrates the following advantages: the method is relatively accurate in terms of visual object classification, learning and classification is very fast, expanding the system knowledge is efficient as adding new visual classes to the system requires generation of new fuzzy rules whereas in the case of bag-of-features it requires new dictionary generation and re-learning of classifiers. The paper was published in 2016 in Information Sciences, a very prestigious journal in the area of computer science with the current impact factor of 6.795 and attracted 145 citations.

Publication Summary , Evidence of Impact and Research Monographs

The above list of major publications of Professor Leszek Rutkowski includes only part of his very recent works published or accepted for publication, eight of them in 2020 or 2021. The almost complete lists are available here:

a) https://scholar.google.com/citations?hl=en&user=DrCG7jAAAAAJ&view_op=list_works&sortby=pubdate
b) https://dblp.org/pid/99/3158.html

The list of 10 candidate’s major publications does not include his three individual monographs, having a few thousand citations, with the foreword or preface from top world researchers:

1) Flexible Neuro-Fuzzy Systems, Kluwer Academic, 2004, foreword by Lotfi Zadeh as follows:

"To write a foreword to Professor Rutkowski's opus "Flexible Neuro-Fuzzy Systems" or FNFS for short, was a challenging task. Today, there exists an extensive literature on neuro-fuzzy systems, but Professor Rutkowski's work goes far beyond what is in print. FNFS ventures into new territory and opens the door to new directions in research and new application areas.

To say that Professor Rutkowski's work is a major contribution to the theory and application of neuro-fuzzy systems is an understatement. The wealth of new ideas, the thoroughness of analysis, the attention to detail, the use of computer simulation, the problems at the end of each chapter, and high expository skill, combine to make Professor Rutkowski's work a must reading for anyone interested in the conception, design and utilization of intelligent systems. Professor Rutkowski and the publisher, Kluwer, deserve a loud applause".

2) New Soft Computing Techniques for System Modelling, Pattern Classification and Image Processing, Springer, 2004, preface by Shun-ichi Amari as follows:

"Soft computing is a key concept for the creation of such human-friendly technology in our modern information society. Professor Rutkowski is a pioneer in this field, having devoted himself for many years to publishing a large variety of original work. The present volume, based mostly on his own work, is a milestone in the development of soft computing, integrating various disciplines from the fields of information science and engineering.

The book has clear motivation to give strong impact to the field of soft computing. It is my pleasure to feel sympathies with the enthusiasm and energy of Professor Rutkowski to devote himself to the new emerging field."

3) Computational Intelligence, Springer, 2008, foreword by Jacek Zurada (this book was published in three languages: Polish, English and Russian) as follows:

"Publication of this book is a special event. This valuable title fills a serious gap in domestic science and technical literature. At the same time it introduces a reader to the most recent achievements in the quickly developing branch of knowledge which the computational intelligence has been for several years. The field, which is a subject of this book, is one of t hose important fields of science which enable to process information included in data and give their reasonable interpretation programmed by a user.

It should be noted Professor Leszek Rutkowski has been included in the list of the top 2% of most influential world researchers https://elsevier.digitalcommonsdata.com/datasets/btchxktzyw/3. Moreover, with the h – index equal to 42 in the Web of Science and 43 in the Scopus, he is one of the most cited researchers in computer science in Poland.