JVM Memory increase drastically

Hi ...

I have a java application named seatunnel....
we are currently using it....whenever i run the cluster the jvm takes some space to execute....after execution the jvm does not releases the space...
how can i resolve this issue....
I have used visualvm to see how much space is allocated to the application.
it takes the heap size as
Size: 2,147,483,680 B
Max: 8,589,934,624 B
and the size of jvm as seen in the cmd using bpytop it increases gradually
what to do can anyone suggest??

The application name that I am using is Apache Seatunnel which is an ingestion tool which ingests data from one source to many different sources...this tool is available in web.....I am using this tool to for ingesting the data. Apache Seatunnel basically built on java...and so we have created jobs for eg to ingest data from mysql to mysql and other variety of jobs as well...Now to run these jobs parallelly and anytime we are running a seatunnel cluster on which these jobs gets ran...

so initially the cluster size is 400mb.....after the 1st job run the size taken by the cluster increases to 800mb it may vary depending on the table size or the data size that we are transferring....then cluster does not releases the memory that it has occupied....after 2nd job run the cluster again grows to 1.5gb....the cluster size keep on increasing...and it never releases the memory....we tested out upto 4.5gb the size of the cluster increases and after the job completion it does not release the memory....and subsequently the size of the main memory goes to around 800mb.....after again if we try to run any job the cluster gets crashed....or comes out as it requires more memory and is not able to fulfill it space requirement as the main memory gets to 100mb or 14mb..... after the cluster crashes....the main memory regains its original space taken by the cluster....so it becomes around 5gb.....we have not tested for more than 5gb of space....

what can we conclude from this.....can we free the space taken by the cluster as shown in the screenshot of cpu utilization....or can we write an external code in java to release the space..... The Apache seatunnel has provided .sh file to start the cluster and to stop the cluster...but it is not a good practice to start and stop the clusters whenever needed or required....

Welcome to the Ranch

We would need lots more information about what you are doing in your application before we can help. What happened later on?

Parul Shrikhande wrote:what to do can anyone suggest??

Since you are running Visual VM, I suggest using the Profiler and Sampler to understand what is filling-up the heap.  Pay attention to the Generations column, which indicates the number of times the objects have survived a garbage collection.

You could also take a snapshot/dump of the heap (before it gets too large), and then use the various views to see what types of objects are in the heap, their sizes, counts, etc.

Parul Shrikhande wrote:The application name that I am using is Apache Seatunnel...

Make sure to check the Seatunnel project issues to see if the problem you are experiencing is already known/reported.

Maybe this one?

JVM Memory Increase in Seatunnel #6786 wrote:
https://github.com/apache/seatunnel/issues/6786
I am running the seatunnel cluster continuously, we are running the seatunnel on ec2 and after each run of a job for eg jdbc to jdbc the jvm size increases,....after each run the size increases continuously untill the memory in the ram is being occupied....if more space is required...it crashes the job...