Hi support team,

Currently we faced with the problem of deployment process. Whenever we deploy a .iar file to the engine (via adminUi, or ivy_deployment_tool), it will cost us a lot of time (about 5-10 minutes).

But after we restart, the first time deployment is much much faster (take around 1 minutes).

Is there any hint for me to improve the deployment without restart engine?

The ivy version: Powered by Axon.ivy Engine 7.0.11.1903290958 Copyright © 2001-2019 AXON IVY AG

Thank you!

asked 07.11.2019 at 22:23

khanh11166's gravatar image

khanh11166
515712
accept rate: 0%

edited 08.11.2019 at 03:54

Denis%20Huelin's gravatar image

Denis Huelin ♦
665

Hard to say what causes the delays. But the deployment process definitively shouldn't take that long unless your diskspace is almost full or CPU is under constant stress. Maybe the log outputs would help to see which stage takes that long (validation or a certain deployment step?) Can you share the size of the IAR you are trying to deploy as well?

(08.11.2019 at 05:24) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

The file size is depend on module I deploy, sometime the file is just small (<1MB), some other is bigger (~50 MB). But the time is the same same with both 2 of them.

The CPU and RAM is not always 100% at the deploy time. So I'm not sure it related to the issue.

(10.11.2019 at 23:08) khanh11166 khanh11166's gravatar image

ok then, but as said there is no known performance issue at deploy time. so, it's your turn to provide more insights on your actual deployment. hard to advise anything with the facts currently available

(11.11.2019 at 03:08) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

Sorry for late reply, today we faced the problem again, and it consume almost all CPU. Could you take a look on the picture and share me some idea?

One question, could I increase the heap will help?!

(28.11.2019 at 05:00) khanh11166 khanh11166's gravatar image

the screenshot shows that the CPU is high. but still we have no facts what is happening. What would help to nail the issue:

  1. logs from the engine during that time
  2. visual vm inspection: thread dumps, gc activity screenshots https://developer.axonivy.com/doc/latest/engine-guide/monitoring/visualvm.html (yes, maybe the heap is an issue, but how should we know without jvm inspection?)
  3. or even better: flight recorder records taken during these deployments https://docs.oracle.com/javacomponents/jmc-5-4/jfr-runtime-guide/about.htm#JFRUH172
(28.11.2019 at 05:31) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

I tried to record by JMC, and found out that https://prnt.sc/q6etrn when unzip the iar file, it took too long....

more details: I'm, using window server

(05.12.2019 at 00:00) khanh11166 khanh11166's gravatar image

the screenshot shows that CPU is blocked while using a third-party for deploying ('soreco deployer'). this is not part of the axonivy.core and may has its own well known quality issues. Moreover, its feature-set is covered natively by the platform. So I'd advise you to switch to a standard deployment approach: https://developer.axonivy.com/doc/7.0.latest/EngineGuideHtml/administration.html#administration-deployment-directory

(05.12.2019 at 02:48) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image
showing 5 of 7 show 2 more comments

Since unzipping is slow I assume that your operating system slows it down: I'd look for involved virus scanners or a crypted filesystem.

Nevertheless, I'd recommend to measure deployment also with a standard deployment approach which is actually supported by the ivy.core platform: https://developer.axonivy.com/doc/7.0.latest/EngineGuideHtml/administration.html#administration-deployment-directory

link

answered 05.12.2019 at 02:51

Reguel%20Wermelinger's gravatar image

Reguel Werme... ♦♦
9.4k21858
accept rate: 70%

edited 05.12.2019 at 02:54

a glimpse towards 8.0 LTS also shows that deployment scenarios supported by the platform were remarkably extended: https://developer.axonivy.com/doc/8.0.0/engine-guide/administration/deployment.html

... one benefit is for instance, that IAR projects being deployed no longer need to be unzipped at all. They will run in packed format similar as JAR files do in the JRE.

(05.12.2019 at 03:03) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

Hi Reguel, We have changed to use linux for the engine. At the very clean engine (I meant after finish set it up and running), the deploy process is very fast (about maximum 3 minutes). But it keep slower and slower. Now, the same deploy process took around 10 minutes.

We have another engine (running the same physical server with the one above), still have fast deploy.

When I compared the 2 databases:

  • The slow one: public.iwa_businessdata ~ 6000 records, about 6.5GB
  • The fast one: public.iwa_businessdata ~ 300 records, about 150MB (because it use for just a small group of people)

Question is: Is there any step in the deployment process of IVY engine related to public.iwa_businessdata table or other table?

Thanks in advance,

DK

link
This answer is marked "community wiki".

answered 22.04 at 05:14

khanh11166's gravatar image

khanh11166
515712
accept rate: 0%

edited 22.04 at 05:17

That sounds interesting. But I couldn't quickly see any IWA_BusinessData interactions when looking at our deployment process.

However, it should be really simple to analyze what the java process is doing in these 10 mins:

(22.04 at 05:56) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

Thanks Reguel, I will record the JMC and analyze it. Btw, I also tried something below: - Delete all data in iwa_businessdata and iwa.tasks -> still slow - Check the deployment log, seem like the slow part is here: https://prnt.sc/s6i2vz

I will back to you later with the JMC recorded

(27.04 at 00:14) khanh11166 khanh11166's gravatar image

Thanks for the update @khanh11166

The 'DeploymentTool' can of course be a cause of inefficient deployments. As stated before, it's not an official platform tool. Has not been written by members of the ivy-core and is therefore neither supported nor promoted to be used. So if this DeploymentTools is slow - we can't support you in fixing it. Even with the JMC records.

What I still recommend is to migrate to Ivy 8.0 and benefit of all the new cool deployment features natively provided by the platform - without using any unsupported third parties.

(27.04 at 02:29) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image

Thanks for your recommend to use Ivy 8.0, it will be in the future near.

I just finished recorded the CPU sampling (using Java VisualVM - can not use JMC because we are using OpenJDK and OpenJDK not include the FlightRecorder feature). I see that on the slow engine, deployment process run some methods as the picture. https://prnt.sc/s792xy

On the fast engine, the deployment process did'n run these methods. Do you have some idea?

(28.04 at 04:57) khanh11166 khanh11166's gravatar image

Also on linux you can opt in for an Oracle JDK 1.8 and you will be able to use flight recorder.

I do not see something special. Seems like the normal annotation processing work is going on (to detect REST endpoints).

(29.04 at 03:05) Reguel Werme... ♦♦ Reguel%20Wermelinger's gravatar image
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "Title")
  • image?![alt text](/path/img.jpg "Title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Tags:

×44
×26

Asked: 07.11.2019 at 22:23

Seen: 645 times

Last updated: 29.04 at 03:05