LuoJiaAI: A cloud-based artificial intelligence platform for remote sensing image interpretation

ABSTRACT The rapid processing, analysis, and mining of remote-sensing big data based on intelligent interpretation technology using remote-sensing cloud computing platforms (RS-CCPs) have recently become a new trend. The existing RS-CCPs mainly focus on developing and optimizing high-performance data storage and intelligent computing for common visual representation, which ignores remote sensing data characteristics such as large image size, large-scale change, multiple data channels, and geographic knowledge embedding, thus impairing computational efficiency and accuracy. We construct a LuoJiaAI platform composed of a standard large-scale sample database (LuoJiaSET) and a dedicated deep learning framework (LuoJiaNET) to achieve state-of-the-art performance on five typical remote sensing interpretation tasks, including scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction. The details of the LuoJiaAI application experiment can be found at the white paper for LuoJiaAI industrial application. In addition, LuoJiaAI is an open-source RS-CCP that supports the latest Open Geospatial Consortium (OGC) standards for better developing and sharing Earth Artificial Intelligence (AI) algorithms and products on benchmark datasets. LuoJiaAI narrows the gap between the sample database and deep learning frameworks through a user-friendly data-framework collaboration mechanism, showing great potential in high-precision remote sensing mapping applications.

With the development of technologies, including big data, cloud computing, and Artificial Intelligence (AI), Deep Learning (DL) has made remarkable progress in numerous fields, including image recognition, drug discovery, natural language processing, and self-driving cars, since the ImageNet Challenge in 2012 (LeCun, Bengio, and Hinton 2015;Russakovsky et al. 2015;Rao and Frtunikj 2018;Chen et al. 2018;Young et al. 2018).In the remote sensing field, researchers integrate DL with remote sensing interpretation technology and apply them in large-scale Earth observation data, which has achieved remarkable results in various image interpretation tasks, such as scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction (Zhu et al. 2017;Sun et al. 2022;Yuan et al. 2020;Li, Huang, and Gong 2019;Ma et al. 2019).Current DL frameworks, such as PyTorch (Paszke et al. 2019), TensorFlow (Abadi et al. 2016), PaddlePaddle (Ma et al. 2019), andMindSpore (Huawei Technologies Co. 2022), are employed in VHR image interpretation applications for AI models designing, training, and deploying through a high-level programming interface.Some RS-CCPs, such as Google Earth Engine and PIE-Engine, have involved these DL frameworks in increasing the intelligence level for processing and analyzing remote sensing big data.
The key components of RS-CCPs are available databases, Application Programming Interfaces (APIs) provided by computational frameworks, and computational infrastructures.Although many academic institutions and corporations have developed RS-CCPs and promoted numerous critical achievements in Earth AI Sciences, there is a lack of a standard large-scale sample database for image intelligent interpretation tasks that allows users to efficiently retrieve, access, label, and share data.Besides, the computational frameworks used in existing RS-CCPs mainly focus on developing and optimizing high-performance data storage and computing in the perspective of common visual representation, which ignores remote sensing data characteristics such as large image size, large-scale change, multiple data channels, and geographic knowledge embedding, thus impairing the computational efficiency and accuracy.
To solve the above limitations in existing RS-CCPs, we construct a LuoJiaAI platform that brings Huawei's massive data storage and computational capabilities for intelligent interpretation and analysis of remote sensing data to achieve state-of-the-art performance on various image interpretation tasks.In summary, we make the following contributions: As the first contribution of LuoJiaAI, LuoJiaSET builds a large-scale sample database with a unified and scalable classification scheme, an Internet-based crowdsourcing sample collection tool with humanmachine interaction, and a sample platform (Homepage, http://58.48.42.237/luojiaSet) for better data preparation and sharing in remote sensing intelligent interpretation.
The second contribution of LuoJiaAI is designing and implementing a dedicated DL framework called LuoJiaNET with remote sensing data characteristics embedded into the underlying system to improve computational efficiency and performance accuracy (Source code, https://github.com/WHULuoJiaTeam/luojianet).LuoJiaAI also develops a front-end application platform (Homepage, http://58.48.42.237/luojiaNet) with a visual modeling tool that provides online creating, training, and deploying models for improving users' productivity.In addition, there is a growing trend for Earth AI Sciences to create libraries of standardized and benchmark datasets.These benchmark datasets can facilitate users to evaluate developed models on a common and standardized dataset efficiently.LuoJiaAI supports the latest Open Geospatial Consortium (OGC) standards in geographic information and service interfaces (https:// www.ogc.org/projects/groups/trainingdmlswg) for better data sharing and model developing (Yue et al. 2022).
LuoJiaAI accelerates the integration of AI algorithms and RS-CCP, and promotes the sharing and development of standardized data, algorithms, and products, showing great potential in rapid intelligent processing and analysis of remote sensing big data.
The rest of this paper is organized as follows.Section 2 reviews related works on RS-CCPs, sample databases, and deep learning models and frameworks.Section 3 illustrates the platform overview briefly.Section 4 first demonstrates how to organize and build a standard large-scale sample database, Internet-based sample collection, and sample sharing platform.Section 5 introduces the construction method of a dedicated framework with remote sensing data characteristics and a visual modeling tool based on graphical user interface.Section 6 briefly reviews some application experiments in typical image intelligent interpretation tasks.Section 7 provides the conclusions and further research directions for improvement.

Remote sensing cloud computing platforms
RS-CCPs break through the bottleneck of traditional remote sensing technology and promote the technological innovation of data processing and analysis by combining huge data resources with highperformance cloud servers (Amani et al. 2020).Users can only focus on developing the fundamental theories and application models of remote sensing without having difficulties obtaining data and computing resources, thus significantly improving the efficiency of massive data processing and information mining.The main components of existing RS-CCPs include: (  (Maggiori et al. 2017), change detection sample datasets, e.g.HRSCD (Daudt et al. 2019) and WHU-BCD (Ji, Wei, and Lu 2018), and multi-view 3D reconstruction sample datasets, e.g.WHU-MVS (Liu and Ji 2020) and US3D (Marc et al. 2019).Although many RS-CCPs have included the publicly released datasets in their sample databases, it remains several problems in remote-sensing intelligent interpretation: (1) The existing classification schemes of current available sample databases are not in a standardized and unified manner (Tenopir et al. 2020).For example, there are numerous classification schemes for land-use classification tasks.It is often difficult to harmonize these classification schemes due to the heterogeneity of data attributes such as names, resolution, projection, observation time, data format, and semantics of classes.(2) A large-scale sample database with different data distributions is the key factor for improving the performance and generalization of DL models in remote sensing intelligent interpretation (Li and Hsu 2020).However, the limited number and variety of data in existing RS-CCPs make it hard to obtain a good generalization approach to make DL models fit beyond the training dataset.
(3) The majority of an image interpretation task is typically spent on data preparation (Reichstein et al. 2019).Acquiring a large-scale sample database is very costly as labeling is usually manually done by in-house labor.A common solution is to crowdsource hand labeling tasks using services like Amazon Mechanical Turk.However, this leaves nonprofessional users with the time-consuming and difficult task of locating, labeling, and integrating disparate datasets.In the remote sensing field, a lack of an open data labeling and organization platform for large-scale and diverse sample database is an obstacle to realizing efficient image interpretation in geoscience applications.

Deep learning models and frameworks for remote sensing
Convolutional Neural Networks (CNNs) have been demonstrated to be strong and achieved state-of-art performance in many vision tasks (e.g.image classification, object detection, and semantic

Platform overview
Figure 2 shows a platform overview of LuoJiaAI.LuoJiaAI consists of a standard large-scale sample database (LuoJiaSET) and a dedicated DL framework (LuoJiaNET) for five typical remote sensing interpretation applications with a high-performance cloud storage and computation service.The platform is accessed and controlled through an Internetaccessible API and an associated user-friendly interactive development environment that enables rapid data processing and visualization of results.
The LuoJiaSET houses an extensive publicly available sample database for image interpretation, including remote-sensing sample datasets from various satellite and aerial imaging systems in optical and non-optical data types.All of these sample datasets are well organized in LuoJiaSET by a standard large-scale database that allows efficient retrieval, access, label, and share, removing the barriers associated with data management and computing.Users can access and analyze data using a library of operators provided by the data service interface and web console with OGC standard from the LuoJiaSET application layer.
The LuoJiaNET develops a dedicated framework and optimization method for exploiting remote sensing data characteristics (e.g.large image size, large-scale change, multiple data

LuoJiaSET
In remote sensing image intelligent interpretation, superior performance comes from large amounts of standard and high-quality labeled remote sensing data (Li et al. 2020;Van Etten, Lindenbaum, and Bacastow 2018;Sumbul et al. 2019).LuoJiaSET constructs a standard large-scale sample database with a unified and scalable classification scheme for multi-type sensors, multi-temporal and multi-scale remote sensing data (Section 3.1), and develops an Internet-based crowdsourcing sample collection tool and a sample sharing platform that provide strong data support for high-precision remote sensing mapping applications (Section 3.2).

Standard large-scale sample database
This section highlights the building steps of a standard large-scale sample database that supports multidimensional global retrieval and data analysis.

Sample model design
LuoJiaSET defines the corresponding sample types for typical remote sensing interpretation tasks, such as scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction.Figure 3 demonstrates the logical model of all sample types.In this model, the ground object samples are organized by planar images (e.g.singlephase and change detection samples) and multi-view stereo images.Different samples can have the same ground object, and samples from different application tasks can be related in accordance with their geographical coordinate.Each application task can include remote sensing data from different sensor types.

Unified and scalable classification scheme
LuoJiaSET builds a unified classification scheme that supports flexible and extended new categories for the sample database.The sample classification framework is shown in Figure 4.The main idea of this construction strategy is as follows: The whole samples in the database are divided into two concept hierarchies based on the national geographic information classification scheme (GB/T 30322).The ground cover category is defined by the dichotomy in the first classification level.For the second classification level, geographic attributes are added to existing categories for defining new classifications.The concepts of artificial-related ground objects in a two-layer classification scheme are extended by the definition of moving and fixed objects in the object detection task, and a new scalable classification scheme is formed for the whole sample datasets.

Sample data integration
Figure 5 demonstrates the technical process of multisource sample data integration.The original classification scheme in existing open-source datasets should be reformed and mapped to a new unified and scalable classification scheme adopted in LuoJiaSET.All sample data are transformed under the unified encoding rules and relational database structure designed by the LuoJiaSET sample database, and global queries are created for multi-dimensional retrieval and data analysis.The information of all sample datasets, including metadata, attribute table, classification scheme, and copyright, is stored in their corresponding metadata table to facilitate traceability, quality control, and ownership identification.

Internet-based sample collection and sharing service
This section highlights the essential components of the sample sharing platform, including the crowdsourcing sample collection platform with human-machine interaction, and the primary infrastructure layers and high-level application Programming Interfaces (APIs).

Crowdsourcing sample collection with human-machine interaction
LuoJiaSET develops an Internet-based sample collection platform that supports the functions of crowdsourcing collection, online verification, and dynamic sample expansion.In the remote sensing field, label sampling is costly, laborious, and has a high demand for specialized geographic knowledge for sample annotators.Therefore, we propose an automatic/semiautomatic combined labeling tool to improve the efficiency and accuracy of label sampling.The main purpose of this tool is to allow participants to interact with automatic interpretation results obtained from trained network models and attempt to reduce recognition errors.The samples that meet the quality requirement after being verified online by professional staff will be accepted by the sample database.Figure 6 shows the whole technical flow diagram of this strategy.

Sample sharing platform
LuoJiaSET develops an open-source sample sharing platform for remote sensing interpretation, which offers the function of global queries for multidimensional retrieval, statistical analysis of data, and data sharing services.Figure 7 shows the overall framework of this platform.The infrastructure layer is constructed by distributed storage systems, DL Graphical Processing Unit (GPU) clusters, file systems, database systems, and a modern network environment from Object Storage Service of Huawei's AI Computing Center in Wuhan.The abilities of storage, expansion, maintenance, and copyright protection of multi-source remote sensing data are supplied in the data storage layer.The service layer provides the APIs for data access and operation and supports multidimensional semantic queries and data delivery service.The high application layer provides users with more advanced online functions in data access and analysis, such as spatiotemporal data mining, statistical analysis of sample attributes, and knowledge discovery in the whole database.

LuoJiaNET
The DL framework is the crucial infrastructure to connect the hardware, software, and AI applications (Liermann 2021;Chen 2021).LuoJiaNET designs the dedicated DL framework with a Graph-Model-Spectral collaboration core layer for embedding remote-sensing data characteristics (e.g.large image size, large-scale change, multiple data channels, and geographic knowledge embedding) into the underlying system.LuoJiaNET's simplified system architecture is shown in Figure 12.The layer above the core of LuoJiaNET is front-end applications and user interfaces, and the hardware device and data access layer are under the core of LuoJiaNET.The core layer of LuoJiaNET includes Intermediate Representations (IRs), computational graph optimization, execution engine, memory management, visual optimization, security protection, remote sensing data characteristics, and data-framework collaboration.Section 4.1 illustrates the graph-based core framework modules of LuoJiaNET.Section 4.2 demonstrates the Model-Spectral driven data processing module to satisfy the demands for remote sensing characteristics embedding.Section 4.3 introduces the overall architecture design of LuoJiaNET's front-end layer.

Graph based core framework modules
The core modules of LuoJiaNET are employed to build the Directed Acyclic Graphs (DAGs) for data processing.The neural networks can handle input remote sensing data from multiple layers, integrate remote sensing features into data flow and control flow, and implement data-framework collaboration through DAGs representation.Specifically, the core modules of the dedicated DL framework include the  construction of network operators and device management, IRs and their compilation optimization, and automatic differentiation, and modules with remotesensing data characteristics, such as memory scalability for large-scale images and adaptive optimization methods for optimizing multi-scale and multi-channel image features.
In the LuoJiaNET framework, the hardware module provides a unified interface and shields the underlying hardware differences (e.g.CPU, GPU, and NPU).The data organization and management module is applied to achieve high-efficiency storage and read/write for the sample database.The data organization and management module takes charge of dataframework collaboration tasks, including automatic data augmentation and refinement in LuoJiaSET's database.LuoJiaNET has compiled and tested the Geospatial Data Abstraction Library (GDAL) for reading and writing various raster and vector data formats under different operating systems.Without introducing any closed-source projects, the GDAL enables LuoJiaNET to have a controllable and efficient Input-Output (I/O) module to process multi-source remote sensing data.

Model-spectral driven data processing module
This section highlights the strategies of remotesensing data characteristics embedded in the  LuoJiaNET framework, including a memory scalability module and a parallel computing strategy for largescale image processing, the adaptive optimization method for adaptive learning multi-scale and multichannel image spectral features, and an auto-parallel strategy for embedding geographic knowledge.The experiments demonstrate the effectiveness of the proposed data processing modules.

Indirect buffer group mapping.
LuoJiaNET proposes a memory scalability module to construct the correlation buffer areas of local features and global spatial context for supporting large-scale image loading, mapping, and learning.Figure 13  Table 1 reports the mean Intersection over Union (MIoU) results for comparison of the patch-based processing and IBGM method on Gaofen Image Dataset (GID) using the HRNetV2 model (Sun et al. 2019).As shown in Table 1, the network with IBGM method outperforms the traditional patch-based processing method by 1.25% in overall MIoU. Figure 14 visualizes the prediction results on the validation set, which proves that global-local context information is essential.The proposed IBGM method supports largescale image loading and training, and outperforms the traditional patch-based image processing method.

Distributed operator decomposition.
LuoJiaNET proposes a distributed parallel computing strategy based on the idea of operator equivalent  decomposition to handle large-scale image calculation in neural networks (Figure 15).In this strategy, the large-scale input images need to be cut and distributed to multiple training cards.The networks, such as the fully convolutional network (FCN) model (Long, Shelhamer, and Darrell 2015), need to be reformed through IRs and compiler optimization to have the ability to calculate large-scale images in the LuoJiaNET framework directly.The core of this method is to decompose the original large-scale input feature into several small pieces of features for network computing.These small features are combined and recovered to the original size by using image stitching technology, making the computational results equivalent to the initial large-scale input size calculation.Hence, one training card only needs to take charge of a small piece of the large-scale input image, thereby vastly reducing the requirements for device memory.
Table 2 reports the overall MIoU for comparison of the patch-based processing and distributed parallel computing strategy on GID using the FCN8s model.As shown in Table 2, large-scale image input with distributed parallel computing strategy outperforms the traditional patch-based processing    method by 5.05% in overall MIoU. Figure 16 visualizes the prediction results on the validation set.This result proves that the proposed distributed parallel computing strategy enables the network to directly process large-scale images and improve the extraction accuracy by learning larger spatial contexts of images.

Adaptive optimization 5.2.2.1. Adaptive scale memory pool.
LuoJiaNET a scale memory pool for adaptive storing, learning, and optimizing multi-scale stacked image features (Figure 17).This method's key technologies include 1) Multi-scale image pyramid construction.Similar to the Laplacian scale space construction, the high-order pyramid is introduced to reduce the loss of scale variation in building an image pyramid.2) Scale memory pool construction.For each LCP unit image feature (Section 4.2.1), its current-scale, previousscale, and next-scale images are all stored in the scale memory pool.In each network training iteration, the scale memory pool stacks multiple-scale features and sends them to a DL network for adaptive learning.

Dynamic feature selection filter.
LuoJiaNET proposes an adaptive optimization method based on a dynamic feature selection filter to learn and optimize data channels adaptively.As shown in Figure 17, this optimization method uses a fixed memory array size to dynamically store and update the best features in end-to-end network training.The similarity between predicted value and label is the criterion for introducing the high-quality features for each ground object in the dynamic feature selection filter.These highquality features will be applied as additional weights to guide network training.
Table 3 reports the overall OA and Kappa for validating the adaptive optimization method on the WHU-Hi dataset (Zhong et al. 2020) using the FreeNet model.As shown in Table 3, the network  with adaptive optimization is improved by 1.54% in OA and 1.88% in Kappa.Figure 18 visualizes the prediction results on the validation set.This result proves that the proposed adaptive optimization method can bring performance improvements for network models.

Parallel geographic knowledge embedding
LuoJiaNET proposes an auto-parallel computing strategy embedding geographic knowledge for processing different types of remote sensing images (e.g.multi-spectral, hyperspectral, and SAR).This prior knowledge can be integrated into the cost function of the LuoJiaNET framework in multi-dimensional mixed auto-parallel computing by developing the specific operators for prior geographic knowledge extraction (e.g.image texture, boundary, and physics-based feature extraction), thereby achieving the optimal computational assignment of DAGs on multiple hardware devices.Figure 19  In the experiment, we choose the gray-level cooccurrence matrices (GLCMs) (Ojala, Pietikäinen, and Harwood 1996) of remote sensing images as the prior geographic knowledge.Table 4 reports the top-1 accuracy and top-5 accuracy for comparison of nonparallel training, auto-parallel training with GLCM image features, and auto-parallel training with GLCMs embedded in the cost function on the WHU-RS19 dataset (Xia et al. 2010) using the ResNet-50 model.As shown in Table 4, auto-parallel training with GLCM image features outperforms the nonparallel training strategy by 3% in top1-accuracy.The top-5 and top-1 accuracies of auto-parallel training with GLCMs embedded in the cost function are increased to 99% and 87%, respectively.This result proves that the proposed parallel optimization method can significantly improve network accuracy.

Framework sharing platform
This section highlights the technology stack of LuoJiaNET's front-end application architecture.The human-machine interactive modeling tool on the LuoJiaNET platform is introduced to improve users' productivity.

Front-end application architecture
LuoJiaNET's front-end application architecture has three browser/server architecture-based layers, including the interface layer, application layer, and data layer (Figure 20).Supported by Ascend APU DL server of Huawei's AI Computing Center in Wuhan, the LuoJiaNET platform deploys training services in the cloud to provide operator-friendly applications and efficient background computing service for users.The LuoJiaNET platform links the Cesium web map (https://cesium.com/)with the model prediction results through their geographical coordinates, thereby enabling users to view interpretation results in the front-end map window directly.As shown in Figure 21, the LuoJiaNET sharing service platform's main page is composed of the parameter setting bar and map visualization window.Users can switch the map online and freely view it at any map's zoom level.Users can draw a region on the satellite map and then employ the network models from LuoJiaNET's model database (Figure 22) to accomplish five typical remote sensing intelligent interpretation tasks, such as scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction.The final model prediction results can be directly displayed on the map for quick check and output.

Visual modeling tool
LuoJiaNET develops a visual modeling tool based on GUI to provide data-framework collaboration.The overall architecture of this visual modeling tool is shown in Figure 23.This tool transfers the complex neural network structure into a form of comprehensible DAGs diagram representation to improve users' productivity in building models.The input data are successively transmitted, calculated, and flow into the training or storage terminal of LuoJiaNET through every graph node in DAGs.This tool sets up a technological workflow of online data acquisition, network creating, training, and deployment, which successfully hides the complex programming and computing work in the background services to intensely facilitate nonprofessional users.
The main page of LuoJiaNET's visual modeling tool is shown in Figure 24.The left side is the operator database of various network models.Users can directly drag the operators into the main modeling window to generate the corresponding DAGs node of models.Users can perform actions, such as copying and undoing nodes, by using the toolbar at the top of the main window.The right side of the main page is the model parameter editor

Applications
LuoJiaAI is in used across different remote-sensing interpretation tasks, such as scene classification (Figure 25), object detection (Figure 26), land-use classification (Figure 27), change detection (Figure 28), and multiview 3D reconstruction (Figure 29).The results of these validation experiments prove the effectiveness of LuoJiaAI and provide huge support for further optimizing and improving the LuoJiaAI's overall architecture.

Conclusion and future work
In this paper, we illustrate the design considerations and methods for constructing the LuoJiaAI platform, which is composed of a large-scale sample database with a unified and scalable classification scheme (LuoJiaSET) and a dedicated DL framework with remote sensing data characteristics (LuoJiaNET).It is a cloud-based AI platform with OGC standards for intelligent interpretation and analysis of remote sensing data that brings Huawei's massive data storage and computational capabilities to achieve state-of-theart performance on five typical remote sensing interpretation tasks, including scene classification, object detection, land-use classification, change detection, and multi-view 3D reconstruction.
The RS-CCP is bound to be the long-term topic in remote-sensing interpretation with continuous data and computing power growth.The research and development of LuoJiaAI are still in the infancy stage.For LuoJiaSET, we will continue expanding the sample database's size and improving sharing service platform functions, such as multi-dimensional retrieval and data analysis.For LuoJiaNET, we will further study the strategies for large-scale image processing, optimization methods for IRs with remote sensing data characteristics, data-framework collaboration mechanisms for automatically obtaining high-quality labeled data, and more visual tools and APIs in front-end web service for better human-machine interaction.LuoJiaAI shows a great potential to break the separation of remote sensing data, intelligent interpretation algorithms, and cloud-based computing resources in high-precision remote sensing mapping tasks.

Notes on contributors
Zhan Zhang is a PhD student at the State Key Laboratory of Information Engineering in Surveying, Mapping, and Remote Sensing, Wuhan, China.His research interests mainly include machine learning system, remote sensing intelligent interpretation, and machine vision.He contributed to the LuoJiaAI platform design and integration, LuoJiaNET's remote sensing characteristics module implementation, and manuscript writing of this paper.Xiangyun Hu is a professor and head of the department of photogrammetry with the School of Remote Sensing and Information Engineering, Wuhan University.His research interests mainly include artificial intelligence and pattern recognition, remote sensing software development, and remote sensing intelligent interpretation.He led the WHU-LuoJiaAI Group and contributed to the methodology and implementation of this paper.

Mi
Hanjiang Xiong is a professor in 3-D geographic information system at Wuhan University.His research interests mainly include geospatial data management, 3-D visualization, augmented reality, and indoor and outdoor geographic information system.He contributed to the implementation of this paper.
Huan Zhou is a master's student at the Department of Land-Surveying and Geo-Informatics, Hong Kong Polytechnic University.His research interest is in web server development and cartographic visualization.He contributed to the overall architecture design and implementation of the LuoJiaNET's front-end layer, and use cases of this paper.
Zhipeng Cao is a PhD student at the School of Remote Sensing and Information Engineering, Wuhan University.His research interest is in big spatiotemporal data management.He contributed to the implementation of this paper.
segmentation)(Alzubaidi et al. 2021).Examples include AlexNet(Krizhevsky, Sutskever, and Hinton 2017), VGG-Net(Simonyan and Zisserman 2014), GoogleNet(Szegedy et al. 2015), ResNet(He et al. 2016), DeepLab(Chen et al. 2017), and HRNet(Sun et al. 2019).These common vision models have numerous variants and form a wide variety of advanced CNN-based networks in different remote sensing image interpretation tasks, such as NEX-Net (Anwer et al. 2018), S2A-Net(Han et al. 2021), FreeNet(Zheng et al. 2020), DSIFN(Zhang et al. 2020), and MVS-Net(Yao et al. 2018).On the other hand, neural architecture search has become a new trend in model architecture design, including reinforcement learning strategy, evolutionary algorithm, and gradientbased method(Ren et al. 2021).DL frameworks accelerate the development of artificial intelligence techniques in vision applications by providing convenient high-level interfaces for users to design DL models on different computing devices(Chen et al.  2018).Some RS-CCPs have directly integrated these DL frameworks (like TensorFlow and PyTorch) into traditional modeling workflows to increase the intelligence level.However, the current DL models and frameworks used in geoscience AI research mainly focus on processing natural images (e.g.indoor-outdoor small-size images), which ignore the characteristics of remote sensing data such as large image size, large-scale change, multiple data channels, and geographic knowledge embedding, thus impairing the computational efficiency and accuracy.

Figure 3 .
Figure 3. Logical model of sample type.

Figure 5 .
Figure 5. Flowchart of multi-source sample data integration.

Figure 7 .
Figure 7. Overall framework of the sample sharing service.

Figure 6 .
Figure 6.Technical flow diagram of automatic/semi-automatic combined labeling.
demonstrates the brief procedure of this Indirect Buffer Group Mapping (IBGM) method.The large-scale image is divided into different global correlation buffers based on labeled data.The pointers reserved in global correlation buffer areas are called global indirect buffer (GCP) unit.In the GCP Unit, the technique of quadtree spatial indexing is constructed to obtain context features in each local context processing (LCP) unit.The GCP unit and LCP unit features are concatenated in the memory scalability module and weighted by General Matrix Multiplication unit in the network model to achieve the global-local context representation and learning.

Figure 13 .
Figure13.Process of IBGM method in memory scalability module for large-scale image processing.

Figure 14 .
Figure 14.Visualization of different large-scale image processing methods on GID.

Figure 15 .
Figure 15.Workflow of distributed parallel computing strategy.

Figure 16 .
Figure 16.Visualization of different large-scale image processing methods on GID.

Figure 17 .
Figure 17.Process of adaptive optimization method for optimizing multi-scale and multi-channel image features.
demonstrates this auto-parallel computing strategy.The auto-parallel cost function contains 1) Delay and bandwidth communication cost of transmission operators between devices.2) Operation cost on different devices based on auto-parallel strategy.3) Geographic knowledgebased cost in parallel training.

Figure 19 .
Figure 19.Auto-parallel computing strategy embedding prior geographic knowledge.

Figure 20 .
Figure 20.Overall architecture of the LuoJiaNET's front-end application.

Figure 21 .Figure 22 .
Figure 21.Main page of the front-end application, with object detection taken as an example.

Figure 23 .
Figure 23.Overall architecture of the LuoJiaNET's visual modeling tool.

Figure 24 .
Figure 24.Main view of visual modeling tool.

Figure 25 .
Figure 25.Visualization on AID using ResNet-50 in scene classification task.
Zhang is an associate researcher at the School of Remote Sensing and Information Engineering, Wuhan University.He serves as the chief artificial intelligence scientist in Handleray Corporation and technical director in WHU-LuoJiaAI Group.His research interests mainly include computer vision, machine learning, with particular interest in semantic object segmentation and the construction of DL framework.He contributed to the overall architecture design and implementation of the LuoJiaNET, and manuscript writing of this paper.Jianya Gong is the Academician of Chinese Academy of Sciences, Professor and Dean of School of Remote Sensing and Information Engineering, Wuhan University.His research interests mainly include remote sensing image processing, spatial data infrastructure, geospatial data interoperability, and artificial intelligence.He contributed to the LuoJiaAI project initiation, and implementation of this paper.

. Sample databases for remote sensing
channels, and geographic knowledge embedding) to increase computational efficiency and accuracy in five typical image interpretation tasks.LuoJiaNET also develops a front-end application platform with a visual modeling tool to provide online model services (e.g.model creation, model training, and model deployment) to improve user productivity.Users can sign up for the LuoJiaNET homepage to access the user's guide, tutorials, model examples, function references, and educational curricula.

Table 1 .
Land-use classification results on GID with HRNetV2 model.

Table 2 .
Land-use classification results on GID with FCN8s model.

Table 3 .
Land-use classification results on WHU-Hi with FreeNet model.
Figure18.Visualization of the adaptive optimization method on WHU-Hi.

Table 4 .
Scene classification results on WHU-RS19 with different parallel computing strategies.