4.1.1 Environment

Back to Master

Runtime Information

NameValue
Java Home/usr/lib/jvm/java-21-amazon-corretto.x86_64
Java Version21.0.10 (Amazon.com Inc.)
Scala Versionversion 2.13.17

Spark Properties

NameValue
spark.blockManager.port0
spark.cores.max4
spark.driver.bindAddress0.0.0.0
spark.driver.cores2
spark.driver.host10.0.3.238
spark.driver.memory3g
spark.driver.port0
spark.dynamicAllocation.enabledfalse
spark.eventLog.dirfile:///opt/spark/events
spark.eventLog.enabledtrue
spark.executor.cores2
spark.executor.instances3
spark.executor.memory3g
spark.hadoop.fs.s3a.implorg.apache.hadoop.fs.s3a.S3AFileSystem
spark.history.fs.logDirectoryfile:///opt/spark/events
spark.jars.packagesorg.apache.hadoop:hadoop-aws:3.4.2
spark.masterspark://10.0.3.238:7077
spark.scheduler.modeFAIR
spark.serializerorg.apache.spark.serializer.KryoSerializer
spark.shuffle.service.enabledfalse
spark.sql.adaptive.enabledtrue
spark.sql.files.ignoreMissingFilestrue
spark.sql.shuffle.partitions16
spark.submit.deployModeclient
spark.worker.cleanup.appDataTtl3600
spark.worker.cleanup.enabledtrue
spark.worker.cleanup.interval300

Hadoop Properties

System Properties

Metrics Properties

Classpath Entries

Environment Variables