I spent good amount of time to find out microsoft technology stack to build highly scalable platform Using .NET Core for the backend, Angular for the frontend, and a combination of Postgres and MongoDB for the database can provide a solid foundation for a high-performance and scalable web application.
.NET Core is a modern, open-source framework for building cross-platform applications. It offers high performance, scalability, and security, making it a popular choice for developing backend applications.
Angular is a powerful frontend framework that allows for the development of complex and responsive user interfaces. It is known for its speed and performance, and it offers a wide range of tools and features for building scalable and maintainable applications.
Postgres is a reliable and scalable SQL database that offers high performance and robust features. It is widely used in production environments and offers a range of advanced features such as support for advanced indexing, JSON data types, and full-text search.
MongoDB, on the other hand, is a NoSQL database that offers high scalability and performance. It is designed to handle large volumes of unstructured data, and it offers a flexible data model that allows for easy scaling and data modeling.
By combining these technologies, you can build a web application that offers high performance, scalability, and flexibility. The combination of a robust backend framework like .NET Core, a powerful frontend framework like Angular, and a combination of SQL and NoSQL databases like Postgres and MongoDB can provide a solid foundation for building a modern and scalable web application.
Next challange was how to sync data from Postgres to MongoDB
Syncing data between PostgreSQL and MongoDB can be challenging due to the differences in data structures and storage models between the two databases. However, there are several tools and approaches available to help simplify the process.
One common approach is to use a data migration tool like Apache NiFi or Talend to extract data from PostgreSQL, transform it into a format that can be stored in MongoDB, and then load it into MongoDB. These tools offer a visual interface for defining data flows and transformations, which can help simplify the process of syncing data between the two databases.
Another approach is to use a third-party tool like the MongoDB Connector for BI, which allows you to connect to PostgreSQL as a data source and query the data directly from MongoDB. This approach can be useful if you only need to access data from PostgreSQL occasionally, as it allows you to query the data without having to move it to MongoDB.
In terms of syncing data from multiple PostgreSQL tables to MongoDB, one approach is to denormalize the data and combine it into a single document in MongoDB. This can be done using aggregation pipelines in MongoDB, which allow you to combine data from multiple tables into a single document. Alternatively, you can use a tool like Talend or Apache NiFi to perform the data transformation and denormalization before loading the data into MongoDB.
Overall, while syncing data between PostgreSQL and MongoDB can be challenging, there are several tools and approaches available to help simplify the process. The best approach will depend on the specific requirements of your application and the amount and complexity of the data you need to sync.
Sandeep Palod 