Meraki

Community

Service Graph Connector script include impacts nodes due to high memory consumption

Steelneck
Comes here often
a month ago
a month ago

Service Graph Connector script include impacts nodes due to high memory consumption

We are using Meraki Service Graph Connector 1.4.0 (not upgraded yet due to failed attempts in sub-prod instances)

 

Today our Prod instance turned slow and for some time unreachable. After raising a Support ticket to ServiceNow we learnt that this was due to that our nodes were impacted by high memory consumption.

 

The culprit in this high memory consumption was according to ServiceNow support that due to high memory consumption from the job SG-Meraki Devices causing one of the nodes to restart, causing extra pressure on the other node. On further analysis, the variable networks_array was found occupying high memory from the following script include SGConnectorMerakiImportUtilsBase

 

I could also see that the latest running SGMerakiGetNetworks flow failed and gave the following message:

"Request not sent to uri= https://api.meraki.com/api/v1/organizations/619244948763443737/networks?perPage=1000&startingAfter=L... : org.apache.commons.httpclient.HttpException: Session contains no certificates - Untrusted"

 

I'm not sure if that flow error message is related to the script include failure though.

 

What is important now is that what can we do to avoid this in the future? Is a plugin update needed to solve this script include? Right now, we don't feel to encouraged to continue using the service graph connector. Also, have anyone else got affected by this?

 

----------------------------------

 

Below is the heap dump analysis:

Type   |Name                          |Value
-------------------------------------------------------------------------------------------------------------------------
boolean|fShouldStop                   |false
ref    |fSchedulerThread              |glide.scheduler
boolean|fUsingCentralScheduler        |true
ref    |fLockCurrentJob               |java.lang.Object @ 0x876c3c08
ref    |fBackgroundTransaction        |com.glide.sys.BGTransaction @ 0xa112a8a8
ref    |fCurrentJobExecutor           |com.glide.schedule.JobExecutor @ 0xa107fe80
ref    |fCurrentJob                   |com.glide.schedule.JobDescriptor @ 0xa107fee8
ref    |fScheduleUserDomainId         |null
ref    |fSchedulerUserId              |system
ref    |fLastRunName                  |SG-Meraki Devices
ref    |fStats                        |com.glide.schedule.WorkerStats @ 0x876c3bd8
ref    |fJobReleasedHandler           |com.glide.schedule_v2.SchedulerThread$$Lambda$5087+0x00000001018ff940 @ 0x876c3bc8
ref    |fJobTracker                   |com.glide.schedule_v2.AggregatedJobTrackers @ 0x875f7cf0
ref    |fWorkerTracker                |com.glide.schedule_v2.SchedulerThread$WorkerTracker @ 0x875f7c88
ref    |fQueue                        |com.glide.schedule_v2.queue.MultiLevelPriorityBlockingQueue @ 0x87508258
ref    |fWorkerDescriptor             |com.glide.schedule_v2.scheduler.types.WorkerDescriptor @ 0x876c3ba8
int    |fJobWaitCount                 |4
boolean|fHaveEvaluatedRunCondition    |true
ref    |fManager                      |com.glide.sys.WorkerThreadManager @ 0x827c3810
boolean|fShouldStop                   |false
boolean|fUnstoppable                  |false
int    |threadLocalRandomSecondarySeed|-1417648147
int    |threadLocalRandomProbe        |-1460985905
long   |threadLocalRandomSeed         |-6452299804564959529
ref    |uncaughtExceptionHandler      |null
       |                              |25 out of 43 displayed
-------------------------------------------------------------------------------------------------------------------------



Class Name                                                                               | Shallow Heap | Retained Heap | Percentage
-------------------------------------------------------------------------------------------------------------------------------------
                                                                                         |              |               |           
org.mozilla.javascript.NativeCall @ 0xa143b990                                           |      0.00 MB |     602.60 MB |     43.57%
|- slotMap org.mozilla.javascript.ThreadSafeSlotMapContainer @ 0xa143b9f0                |      0.00 MB |     602.60 MB |     43.57%
|  |- map org.mozilla.javascript.EmbeddedSlotMap @ 0xa143ba08                            |      0.00 MB |     602.60 MB |     43.57%
|  |  |- org.mozilla.javascript.Slot @ 0xa143ba78                                        |      0.00 MB |     602.60 MB |     43.57%
|  |  |  '- value org.mozilla.javascript.NativeArray @ 0xa143baa0                        |      0.00 MB |     602.60 MB |     43.57%
|  |  |     |- dense java.lang.Object[2734845] @ 0xc8144de8                              |     10.43 MB |     602.60 MB |     43.57%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb4de6230                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xaa0da3f8                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xbcdc12f0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xc03f95a0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xaf2d1c30                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xa6d90868                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb73a1170                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb0dbd588                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xc4049ad0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xadb06da8                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb9df96c0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xac5824e8                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xa63f6d48                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xc03f96f0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xbcdc1440                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xac582548                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb4de6380                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xc4049c20                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xa3456c58                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xa91c0178                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb73a12c0                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xaf2d1d50                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xb9df9810                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xc03f9840                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  |- org.mozilla.javascript.NativeObject @ 0xa24aa818                       |      0.00 MB |       0.00 MB |      0.00%
|  |  |     |  '- Total: 25 of 18,48,102 entries; 18,48,077 more                         |              |               |           
|  |  |     |- slotMap org.mozilla.javascript.ThreadSafeSlotMapContainer @ 0xa143bb00    |      0.00 MB |       0.00 MB |      0.00%
|  |  |     '- Total: 2 entries                                                          |              |               |           
|  |  |- org.mozilla.javascript.Slot @ 0xa15784c0                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  |- org.mozilla.javascript.Slot @ 0xa1578328                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  |- slots org.mozilla.javascript.Slot[16] @ 0xa143ba28                              |      0.00 MB |       0.00 MB |      0.00%
|  |  |- lastAdded org.mozilla.javascript.Slot @ 0xa1578288                              |      0.00 MB |       0.00 MB |      0.00%
|  |  |- org.mozilla.javascript.Slot @ 0xa15782b0                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  |- org.mozilla.javascript.Slot @ 0xa15782d8                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  |- org.mozilla.javascript.Slot @ 0xa1578300                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  |- firstAdded org.mozilla.javascript.Slot @ 0xa1578470                             |      0.00 MB |       0.00 MB |      0.00%
|  |  |- org.mozilla.javascript.Slot @ 0xa1578498                                        |      0.00 MB |       0.00 MB |      0.00%
|  |  '- Total: 10 entries                                                               |              |               |           
|  |- lock java.util.concurrent.locks.StampedLock @ 0xa1578258                           |      0.00 MB |       0.00 MB |      0.00%
|  '- Total: 2 entries                                                                   |              |               |           
|- arguments org.mozilla.javascript.Arguments @ 0xa1578140                               |      0.00 MB |       0.00 MB |      0.00%
'- Total: 2 entries                                                                      |              |               |           
com.glide.sys.cache.CacheMan @ 0x816150b0                                                |      0.00 MB |     125.41 MB |      9.07%
com.snc.orbit.container.tomcat8.Tomcat8$OrbitTomcat8ClassLoader @ 0x810dcaa8             |      0.00 MB |      74.07 MB |      5.36%
class org.mozilla.javascript.JavaMembers @ 0x82a93930                                    |      0.00 MB |      39.16 MB |      2.83%
class com.glide.script.Compiler @ 0x816213e0                                             |      0.00 MB |      26.54 MB |      1.92%
org.apache.catalina.webresources.Cache @ 0x81152ce0                                      |      0.00 MB |       9.40 MB |      0.68%
class org.mozilla.javascript.ContextFactory @ 0x8113fcb8                                 |      0.00 MB |       6.84 MB |      0.49%
class com.glide.ui.jelly.GlideJellyContext @ 0x816b6858                                  |      0.00 MB |       6.84 MB |      0.49%
com.glide.schedule_v2.SchedulerWorkerThread @ 0x876acfa0  glide.scheduler.worker.5 Thread|      0.00 MB |       6.40 MB |      0.46%
class org.mozilla.javascript.FunctionObject @ 0x811d8108                                 |      0.00 MB |       6.09 MB |      0.44%
java.util.ArrayList @ 0x81152cc8                                                         |      0.00 MB |       5.62 MB |      0.41%
graphql.schema.dynamic.GraphQLDynamicSchema @ 0x88fffe18                                 |      0.00 MB |       5.17 MB |      0.37%
class com.glide.util.GlideProperties @ 0x81231990                                        |      0.00 MB |       4.41 MB |      0.32%
class com.glide.util.JSONUtil @ 0x853aaa88                                               |      0.00 MB |       4.34 MB |      0.31%
java.lang.Thread @ 0x823547e8  Default-thread-15 Thread                                  |      0.00 MB |       3.48 MB |      0.25%
com.glide.db.DBObjectManager @ 0x9e0e14d8                                                |      0.00 MB |       3.37 MB |      0.24%
java.util.zip.ZipFile$Source @ 0xd03c0070                                                |      0.00 MB |       3.34 MB |      0.24%
int[868768] @ 0xd6c430c8 Unreachable                                                     |      3.31 MB |       3.31 MB |      0.24%
java.util.zip.ZipFile$Source @ 0xd26eace8                                                |      0.00 MB |       3.26 MB |      0.24%
class java.security.Policy @ 0x858d4550 System Class                                     |      0.00 MB |       3.06 MB |      0.22%
java.util.zip.ZipFile$Source @ 0xd5a9d5d0                                                |      0.00 MB |       3.06 MB |      0.22%
com.glide.db.DBObjectManager @ 0x8175bc88                                                |      0.00 MB |       3.04 MB |      0.22%
com.glide.script.GlobalRhinoScope @ 0xa1128890                                           |      0.00 MB |       2.84 MB |      0.21%
com.glide.sys.cache.ConcurrentLFUCache @ 0x8296c8f0                                      |      0.00 MB |       2.78 MB |      0.20%
Total: 24 of 8,15,967 entries; 8,15,943 more                                             |     51.65 MB |   1,383.01 MB |    100.00%
-------------------------------------------------------------------------------------------------------------------------------------



Type |Name       |Value
-----------------------------------------------------------------
ref  |orderedNext|org.mozilla.javascript.Slot @ 0xa15784c0
ref  |next       |org.mozilla.javascript.Slot @ 0xa1578470
ref  |value      |org.mozilla.javascript.NativeArray @ 0xa143baa0
short|attributes |4
int  |indexOrHash|-1571755617
ref  |name       |networks_array
-----------------------------------------------------------------


glide.scheduler.worker.3
  at java.lang.Object.wait(J)V (Native Method)
  at java.lang.Object.wait()V (Object.java:338)
  at com.glide.lock.JVMLockToken.getWithTimeout(JJZ)Z (JVMLockToken.java:68)
  at com.glide.lock.JVMLockToken.get()V (JVMLockToken.java:45)
  at com.glide.lock.JVMLock.get()V (JVMLock.java:33)
  at com.glide.sys.cache.TypeSafeCacheManager.get(Ljava/lang/Object;Ljava/util/function/Supplier;)Ljava/lang/Object; (TypeSafeCacheManager.java:121)
  at com.glide.db.SizeClassCache.get(Ljava/lang/String;)I (SizeClassCache.java:25)
  at com.glide.db.ElementDescriptor.getSizeClass()I (ElementDescriptor.java:3603)
  at com.glide.db.TableDescriptor.getSizeClass()I (TableDescriptor.java:604)
  at com.glide.db.meta.Tables.isLarge(Lcom/glide/db/TableDescriptor;Lcom/glide/db/DBI;)Z (Tables.java:74)
  at com.glide.db.meta.Table.isLarge(Lcom/glide/db/DBQuery;)Z (Table.java:1074)
  at com.glide.db.meta.Table.queryBasic(Lcom/glide/db/meta/Query;Z)V (Table.java:435)
  at com.glide.db.meta.Table.queryGet(Lcom/glide/db/meta/Query;)V (Table.java:284)
  at com.glide.db.meta.Table.query(Lcom/glide/db/meta/Query;)V (Table.java:235)
  at com.glide.script.GlideRecordITable.query()V (GlideRecordITable.java:132)
  at com.glide.script.GlideRecord.query0()Z (GlideRecord.java:3713)
  at com.glide.script.GlideRecord.query()Z (GlideRecord.java:3411)
  at com.glide.script.GlideRecord.queryNoDomain()V (GlideRecord.java:3389)
  at com.glide.script.glide_elements.GlideElementReference.getDisplayValue(Ljava/lang/String;Ljava/lang/String;Ljava/util/Optional;)Ljava/lang/String; (GlideElementReference.java:546)
  at com.glide.script.glide_elements.GlideElementReference.getDisplayValue(Ljava/util/Optional;)Ljava/lang/String; (GlideElementReference.java:411)
  at com.glide.script.glide_elements.GlideElementReference.getDisplayValue()Ljava/lang/String; (GlideElementReference.java:339)
  at com.glide.script.GlideElement.getDisplayValue(I)Ljava/lang/String; (GlideElement.java:1348)
  at com.glide.script.GlideMemoryElement.<init>(Lcom/glide/script/GlideElement;)V (GlideMemoryElement.java:27)
  at com.glide.script.GlideMemoryRecord.setValue(Ljava/lang/String;Lcom/glide/script/GlideElement;)V (GlideMemoryRecord.java:110)
  at com.glide.script.GlideMemoryRecord.loadAll(Lcom/glide/script/GlideRecord;)V (GlideMemoryRecord.java:101)
  at com.glide.script.GlideMemoryRecord.load(Lcom/glide/script/GlideRecord;Ljava/util/ArrayList;)V (GlideMemoryRecord.java:56)
  at com.glide.script.GlideMemoryRecord.<init>(Lcom/glide/script/GlideRecord;)V (GlideMemoryRecord.java:36)
  at com.glide.processors.ProcessorDefinition.build(Lcom/glide/script/GlideRecord;)Lcom/glide/processors/ProcessorDefinition; (ProcessorDefinition.java:102)
  at com.glide.processors.ProcessorRegistry.loadDB()I (ProcessorRegistry.java:114)
  at com.glide.processors.ProcessorRegistry.get()Lcom/glide/processors/ProcessorRegistry; (ProcessorRegistry.java:80)
  at com.glide.script.proxy.HttpClient.isValidAdminProcessor(Ljava/lang/String;Ljava/lang/String;)Z (HttpClient.java:107)
  at com.glide.script.proxy.HttpClient.validateURI(Lorg/apache/commons/httpclient/URI;)V (HttpClient.java:164)
  at com.glide.script.GlideRhinoEnvironment$$Lambda$184+0x00000001005a6938.validateURI(Lorg/apache/commons/httpclient/URI;)V (Unknown Source)
  at com.glide.util.HTTPUtil.validateURI(Ljava/util/List;Lorg/apache/commons/httpclient/URI;)V (HTTPUtil.java:120)
  at com.glide.communications.HTTPClient.validateURI(Lorg/apache/commons/httpclient/URI;)V (HTTPClient.java:626)
  at com.glide.communications.HTTPClient.doExecute(Lorg/apache/commons/httpclient/HttpMethod;Lorg/apache/commons/httpclient/HttpState;)I (HTTPClient.java:544)
  at com.glide.communications.HTTPClient.doExecute(Lorg/apache/commons/httpclient/HttpMethod;)I (HTTPClient.java:515)
  at com.glide.communications.HTTPClient.executeMethod(Lorg/apache/commons/httpclient/HttpMethod;)I (HTTPClient.java:415)
  at com.glide.communications.HTTPRequest.send(Lorg/apache/commons/httpclient/HttpMethod;)Lcom/glide/communications/HTTPResponse; (HTTPRequest.java:186)
  at com.snc.process_flow.operation.FlowHTTPRequest.send(Lorg/apache/commons/httpclient/HttpMethod;)Lcom/glide/communications/HTTPResponse; (FlowHTTPRequest.java:61)
  at com.glide.communications.HTTPRequest.get()Lcom/glide/communications/HTTPResponse; (HTTPRequest.java:67)
  at com.snc.process_flow.operation.HttpOperation.doRequest()Lcom/glide/communications/IHTTPResponse; (HttpOperation.java:251)
  at com.snc.process_flow.operation.AbstractHttpOperation.invoke()V (AbstractHttpOperation.java:236)
  at com.snc.process_flow.operation.RetryableIntegrationOperation.run()V (RetryableIntegrationOperation.java:61)
  at com.snc.process_flow.operation.HttpOperation.run()V (HttpOperation.java:241)
  at com.snc.process_flow.engine.Operation.run2()V (Operation.java:90)
  at com.snc.process_flow.v2.exec.OpRunner.runOperation(Lcom/snc/process_flow/engine/Operation;Lcom/snc/process_flow/v2/model/dto/instruction/Io;Lcom/snc/process_flow/v2/exec/IExecutionContextV2;I)V (OpRunner.java:75)
  at com.snc.process_flow.v2.exec.RetryableIntegrationOpInstructionExecutor.runOperation(Lcom/snc/process_flow/v2/model/dto/instruction/OpInstruction;Lcom/snc/process_flow/v2/exec/IExecutionContextV2;)V (RetryableIntegrationOpInstructionExecutor.java:88)
  at com.snc.process_flow.v2.exec.OpInstructionExecutor.execute(Lcom/snc/process_flow/v2/model/dto/instruction/OpInstruction;Lcom/snc/process_flow/v2/exec/IExecutionContextV2;)Lcom/snc/process_flow/v2/exec/InstructionExecutionResult; (OpInstructionExecutor.java:27)
  at com.snc.process_flow.v2.exec.OpInstructionExecutor.execute(Lcom/snc/process_flow/v2/model/instruction/IInstruction;Lcom/snc/process_flow/v2/exec/IExecutionContextV2;)Lcom/snc/process_flow/v2/exec/InstructionExecutionResult; (OpInstructionExecutor.java:16)
  at com.snc.process_flow.v2.exec.EngineV2.executeInstructions(Lcom/snc/process_flow/v2/exec/IExecutionContextV2;)Lcom/snc/process_flow/v2/exec/EngineV2RunResult; (EngineV2.java:182)
  at com.snc.process_flow.v2.exec.EngineV2.run()Lcom/snc/process_flow/v2/exec/EngineV2RunResult; (EngineV2.java:106)
  at com.snc.process_flow.v2.exec.EngineRunner.lambda$run$0()Lcom/snc/process_flow/v2/exec/EngineV2RunResult; (EngineRunner.java:28)
  at com.snc.process_flow.v2.exec.EngineRunner$$Lambda$9544+0x00000001027f1698.get()Ljava/lang/Object; (Unknown Source)
  at com.snc.process_flow.v2.exec.enviroment.execution.ExecutionScopeRunner.run(Lcom/snc/process_flow/v2/exec/IExecutionContextV2;Ljava/util/function/Supplier;)Ljava/lang/Object; (ExecutionScopeRunner.java:37)
  at com.snc.process_flow.v2.exec.EngineRunner.run(Lcom/snc/process_flow/v2/exec/IExecutionContextV2;)Lcom/snc/process_flow/v2/exec/EngineV2RunResult; (EngineRunner.java:28)
  at com.snc.process_flow.engine.EngineV2Adapter.runWithExecutionResult(Ljava/lang/String;Ljava/util/Map;)Lcom/snc/process_flow/engine/IProcessEngine$IEngineExecution; (EngineV2Adapter.java:180)
  at com.snc.process_flow.engine.EngineV2Adapter.run(Ljava/lang/String;Ljava/util/Map;)Lcom/snc/process_flow/engine/ProcessPlan; (EngineV2Adapter.java:335)
  at com.snc.process_flow.engine.ChildPlanExecutor.executeV2(Lcom/snc/process_flow/engine/plan/executor/ChildPlanExecutorRequest;)Lcom/snc/process_flow/engine/plan/executor/ChildPlanExecutorResult; (ChildPlanExecutor.java:126)
  at com.snc.process_flow.engine.ChildPlanExecutor.executeSync(Lcom/snc/process_flow/engine/plan/executor/ChildPlanExecutorRequest;)Lcom/snc/process_flow/engine/plan/executor/ChildPlanExecutorResult; (ChildPlanExecutor.java:79)
  at com.snc.process_flow.engine.VirtualPagePlanExecutor.executeInCurrentEnvironment(Lcom/snc/process_flow/engine/VirtualPagePlanExecutorRequest;)Lcom/snc/process_flow/engine/plan/executor/ChildPlanExecutorResult; (VirtualPagePlanExecutor.java:103)
  at com.snc.process_flow.engine.VirtualPagePlanExecutor.executeSync(Lcom/snc/process_flow/engine/VirtualPagePlanExecutorRequest;)Lcom/snc/process_flow/engine/plan/executor/VirtualPagePlanExecutorResult; (VirtualPagePlanExecutor.java:53)
  at com.snc.process_flow.engine.VirtualPageFactory.executeChildPlanRequest(Lcom/snc/process_flow/engine/VirtualPagePlanExecutorRequest;)Lcom/snc/process_flow/engine/plan/executor/VirtualPagePlanExecutorResult; (VirtualPageFactory.java:230)
  at com.snc.process_flow.engine.VirtualPageFactory.executeChildPlanV2(Lcom/snc/process_flow/stream/IVirtualPage;I)Lcom/snc/process_flow/stream/VirtualPage; (VirtualPageFactory.java:96)
  at com.snc.process_flow.engine.VirtualPageFactory.getPage(Lcom/snc/process_flow/stream/IVirtualPage;I)Lcom/snc/process_flow/stream/IVirtualPage; (VirtualPageFactory.java:79)
  at com.snc.process_flow.stream.PaginatingDataStream.get()Lcom/snc/process_flow/stream/Record; (PaginatingDataStream.java:144)
  at com.snc.process_flow.stream.PaginatingDataStream.get()Lcom/snc/process_flow/stream/Record; (PaginatingDataStream.java:171)
  at com.snc.process_flow.stream.PaginatingDataStream.hasNext()Z (PaginatingDataStream.java:89)
  at com.snc.process_flow.stream.PaginatingDataStreamProxy.hasNext()Z (PaginatingDataStreamProxy.java:86)
  at com.snc.process_flow.stream.ComplexObjectDataStream.hasNext()Z (ComplexObjectDataStream.java:49)
  at com.glide.plan.runners.scriptable.ScriptableDataStream.hasNext()Z (ScriptableDataStream.java:79)
  at com.glide.plan.runners.scriptable.ScriptableDataStream.jsFunction_hasNext()Z (ScriptableDataStream.java:48)
  at jdk.internal.reflect.GeneratedMethodAccessor1662.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Unknown Source)
  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (Method.java:569)
  at org.mozilla.javascript.MemberBox.invoke(Ljava/lang/Object;[Ljava/lang/Object;)Ljava/lang/Object; (MemberBox.java:236)
  at org.mozilla.javascript.FunctionObject.doInvoke(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/MemberBox;Ljava/lang/Object;[Ljava/lang/Object;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;)Ljava/lang/Object; (FunctionObject.java:674)
  at org.mozilla.javascript.FunctionObject.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;Z)Ljava/lang/Object; (FunctionObject.java:603)
  at org.mozilla.javascript.ScriptRuntime.doCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:2988)
  at org.mozilla.javascript.Interpreter.interpretLoop(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Interpreter$CallFrame;Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1928)
  at org.mozilla.javascript.Interpreter.interpret(Lorg/mozilla/javascript/InterpretedFunction;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1112)
  at org.mozilla.javascript.InterpretedFunction.lambda$call$0(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:161)
  at org.mozilla.javascript.InterpretedFunction$$Lambda$590+0x000000010088bc80.get()Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller$$Lambda$593+0x000000010088ddb0.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.caller.gen.sys_script_include_b778fb601b8d9950263997d8b04bcb43_script.call(Lcom/glide/caller/ICallable;)Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller.call(Ljava/util/function/Supplier;Ljava/lang/String;)Ljava/lang/Object; (ScriptCaller.java:22)
  at org.mozilla.javascript.InterpretedFunction.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:160)
  at org.mozilla.javascript.ScriptRuntime.doCall2(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:3089)
  at org.mozilla.javascript.ScriptRuntime.doCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:2998)
  at org.mozilla.javascript.Interpreter.interpretLoop(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Interpreter$CallFrame;Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1928)
  at org.mozilla.javascript.Interpreter.interpret(Lorg/mozilla/javascript/InterpretedFunction;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1112)
  at org.mozilla.javascript.InterpretedFunction.lambda$call$0(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:161)
  at org.mozilla.javascript.InterpretedFunction$$Lambda$590+0x000000010088bc80.get()Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller$$Lambda$593+0x000000010088ddb0.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.caller.gen.sys_script_include_b778fb601b8d9950263997d8b04bcb43_script.call(Lcom/glide/caller/ICallable;)Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller.call(Ljava/util/function/Supplier;Ljava/lang/String;)Ljava/lang/Object; (ScriptCaller.java:22)
  at org.mozilla.javascript.InterpretedFunction.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:160)
  at org.mozilla.javascript.ScriptRuntime.doCall2(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:3089)
  at org.mozilla.javascript.ScriptRuntime.doCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:2998)
  at org.mozilla.javascript.Interpreter.interpretLoop(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Interpreter$CallFrame;Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1928)
  at org.mozilla.javascript.Interpreter.interpret(Lorg/mozilla/javascript/InterpretedFunction;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1112)
  at org.mozilla.javascript.InterpretedFunction.lambda$call$0(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:161)
  at org.mozilla.javascript.InterpretedFunction$$Lambda$590+0x000000010088bc80.get()Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller$$Lambda$593+0x000000010088ddb0.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.caller.gen.sys_script_include_b778fb601b8d9950263997d8b04bcb43_script.call(Lcom/glide/caller/ICallable;)Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller.call(Ljava/util/function/Supplier;Ljava/lang/String;)Ljava/lang/Object; (ScriptCaller.java:22)
  at org.mozilla.javascript.InterpretedFunction.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:160)
  at org.mozilla.javascript.ScriptRuntime.doCall2(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:3089)
  at org.mozilla.javascript.ScriptRuntime.doCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:2998)
  at org.mozilla.javascript.Interpreter.interpretLoop(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Interpreter$CallFrame;Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1928)
  at org.mozilla.javascript.Interpreter.interpret(Lorg/mozilla/javascript/InterpretedFunction;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1112)
  at org.mozilla.javascript.InterpretedFunction.lambda$call$0(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:161)
  at org.mozilla.javascript.InterpretedFunction$$Lambda$590+0x000000010088bc80.get()Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller$$Lambda$593+0x000000010088ddb0.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.caller.gen.sys_data_source_288088d680821010f8772cdfe9d5f8b5_data_loader.call(Lcom/glide/caller/ICallable;)Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller.call(Ljava/util/function/Supplier;Ljava/lang/String;)Ljava/lang/Object; (ScriptCaller.java:22)
  at org.mozilla.javascript.InterpretedFunction.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:160)
  at org.mozilla.javascript.ScriptRuntime.doCall2(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:3089)
  at org.mozilla.javascript.ScriptRuntime.doCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ScriptRuntime.java:2998)
  at org.mozilla.javascript.Interpreter.interpretLoop(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Interpreter$CallFrame;Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1928)
  at org.mozilla.javascript.Interpreter.interpret(Lorg/mozilla/javascript/InterpretedFunction;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (Interpreter.java:1112)
  at org.mozilla.javascript.InterpretedFunction.lambda$call$0(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:161)
  at org.mozilla.javascript.InterpretedFunction$$Lambda$590+0x000000010088bc80.get()Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller$$Lambda$593+0x000000010088ddb0.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.caller.gen.sys_data_source_288088d680821010f8772cdfe9d5f8b5_data_loader.call(Lcom/glide/caller/ICallable;)Ljava/lang/Object; (Unknown Source)
  at com.glide.script.ScriptCaller.call(Ljava/util/function/Supplier;Ljava/lang/String;)Ljava/lang/Object; (ScriptCaller.java:22)
  at org.mozilla.javascript.InterpretedFunction.call(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (InterpretedFunction.java:160)
  at org.mozilla.javascript.ContextFactory.doTopCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;)Ljava/lang/Object; (ContextFactory.java:719)
  at org.mozilla.javascript.ScriptRuntime.doTopCall(Lorg/mozilla/javascript/Callable;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;Lorg/mozilla/javascript/Scriptable;[Ljava/lang/Object;Z)Ljava/lang/Object; (ScriptRuntime.java:4412)
  at org.mozilla.javascript.InterpretedFunction.exec(Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Scriptable;)Ljava/lang/Object; (InterpretedFunction.java:174)
  at com.glide.script.ScriptCompiler.executeAndPublishMetric(Lorg/mozilla/javascript/Scriptable;Ljava/lang/String;Lorg/mozilla/javascript/Context;Lorg/mozilla/javascript/Script;Lcom/glide/util/StopWatch;)Ljava/lang/Object; (ScriptCompiler.java:77)
  at com.glide.script.ScriptEvaluator.execute(Ljava/lang/String;Lorg/mozilla/javascript/Scriptable;Ljava/lang/String;Lorg/mozilla/javascript/Context;)Ljava/lang/Object; (ScriptEvaluator.java:439)
  at com.glide.script.ScriptEvaluator.evaluate(Lcom/glide/script/fencing/ScriptExpression;)Ljava/lang/Object; (ScriptEvaluator.java:219)
  at com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(Lcom/glide/script/fencing/GlideScriptExpression;Lcom/glide/sys/fencing/Scope;Ljava/util/Map;)Ljava/lang/Object; (GlideScopedEvaluator.java:376)
  at com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(Lcom/glide/script/GlideRecord;Ljava/lang/String;Ljava/util/Map;)Ljava/lang/Object; (GlideScopedEvaluator.java:205)
  at com.glide.script.fencing.GlideScopedEvaluator.evaluateScript(Lcom/glide/script/GlideRecord;Ljava/lang/String;)Ljava/lang/Object; (GlideScopedEvaluator.java:175)
  at com.glide.db.impex.custom.CustomLoader.loadDataFromScript()V (CustomLoader.java:81)
  at com.glide.db.impex.custom.CustomLoader.load()Z (CustomLoader.java:57)
  at com.glide.db.impex.AbstractLoader.startWork()V (AbstractLoader.java:169)
  at com.glide.worker.AbstractProgressWorker.startAndWait()V (AbstractProgressWorker.java:131)
  at com.glide.worker.HierarchicalProgressWorker.startAndWait()V (HierarchicalProgressWorker.java:26)
  at com.glide.worker.AbstractProgressWorker.start()V (AbstractProgressWorker.java:106)
  at com.glide.system_import_set.ImportSetLoader.startWorker(Lcom/glide/worker/AbstractProgressWorker;Ljava/lang/String;)V (ImportSetLoader.java:141)
  at com.glide.system_import_set.ImportSetLoader.startLoader(Lcom/glide/db/impex/AbstractLoader;Ljava/lang/String;Ljava/lang/String;ZLcom/glide/db/impex/ILoaderObserver;)V (ImportSetLoader.java:130)
  at com.glide.system_import_set.ImportSetLoader.loadImportSetTable(Lcom/glide/system_import_set/ConcurrentImportSet;Lcom/glide/script/GlideRecord;Lcom/glide/script/GlideRecord;Lcom/glide/db/impex/datasource/DataSource;Lcom/glide/db/impex/AbstractLoader;)Z (ImportSetLoader.java:114)
  at com.snc.automation.ScheduledImportSetJob.doConcurrentImport(Lcom/glide/script/GlideRecord;Lcom/glide/script/GlideRecord;ILcom/glide/system_import_set/ScheduledImportExecutionContext;)Z (ScheduledImportSetJob.java:290)
  at com.snc.automation.ScheduledImportSetJob.runImport(Lcom/glide/script/GlideRecord;Lcom/glide/system_import_set/ScheduledImportExecutionContext;)Z (ScheduledImportSetJob.java:111)
  at com.snc.automation.ScheduledImportSetJob.runNextImport(Lcom/glide/script/GlideRecord;Lcom/glide/system_import_set/ScheduledImportExecutionContext;)Z (ScheduledImportSetJob.java:86)
  at com.snc.automation.ScheduledImportSetJob.runImport(Lcom/glide/script/GlideRecord;)Z (ScheduledImportSetJob.java:63)
  at com.snc.automation.ScheduledImportJob.execute()I (ScheduledImportJob.java:66)
  at com.glide.schedule.JobExecutor.lambda$executeJob$1(Lcom/glide/job/GlideJob;)Ljava/lang/Void; (JobExecutor.java:195)
  at com.glide.schedule.JobExecutor$$Lambda$5259+0x0000000101982558.call()Ljava/lang/Object; (Unknown Source)
  at com.glide.schedule.JobExecutor.executeJob(Lcom/glide/job/GlideJob;)V (JobExecutor.java:198)
  at com.glide.schedule.JobExecutor.execute(Lcom/glide/job/GlideJob;)V (JobExecutor.java:178)
  at com.glide.schedule.JobExecutor.execute()V (JobExecutor.java:168)
  at com.glide.schedule_v2.SchedulerWorkerThread.executeJob(Lcom/glide/schedule/JobDescriptor;Lcom/glide/sys/BGTransaction;)V (SchedulerWorkerThread.java:609)
  at com.glide.schedule_v2.SchedulerWorkerThread.lambda$process$2(Lcom/glide/schedule/JobDescriptor;Lcom/glide/sys/BGTransaction;)V (SchedulerWorkerThread.java:402)
  at com.glide.schedule_v2.SchedulerWorkerThread$$Lambda$5174+0x000000010192e908.run()V (Unknown Source)
  at com.glide.worker.TransactionalWorkerThread.executeInTransaction(Ljava/lang/Runnable;Lcom/glide/sys/Transaction;)V (TransactionalWorkerThread.java:35)
  at com.glide.schedule_v2.SchedulerWorkerThread.process()V (SchedulerWorkerThread.java:402)
  at com.glide.schedule_v2.SchedulerWorkerThread.run()V (SchedulerWorkerThread.java:178)

 

0 Kudos
1 Reply 1
shwetaP
Meraki Employee
Meraki Employee
2 weeks ago
2 weeks ago

Hi @Steelneck I would recommend upgrading to the latest version (currently on v1.5.1) to resolve this issue. The v1.4 had known bugs related to performance which are resolved in the later releases. 

Get notified when there are additional replies to this discussion.
Welcome to the Meraki Community!
To start contributing, simply sign in with your Cisco ID. If you don't yet have a Cisco ID, you can sign up.
© 2025 Cisco Systems, Inc.