github.com/lyft/flinkk8soperator

What's Changed

• Migrate to GitHub Actions for CI by @danielmmetz in #259
• Support kubernetes 1.22 by @danielmmetz in #258
• Migrate to networking.k8s.io by @anandswaminathan in #265
• Revert "Support kubernetes 1.22 (#258)" by @sethsaperstein-lyft in #280
• [STRMCMP-1639] Integration Test Fix by @sethsaperstein-lyft in #277
• Use Go Modules by @danielmmetz in #274
• update owners by @leoluoInSea in #285
• Monitor Job Vertices State on Deploy by @leoluoInSea in #284
• fix a potential bug by @leoluoInSea in #286
• [STRMCMP-1658] Enable Deploys to Fallback without State by @sethsaperstein-lyft in #287
• [STRMHELP-315] Rollback on Failed Job Monitoring 🐛 by @sethsaperstein-lyft in #291
• [STRMCMP-1659] k8s v1.24 upgrade by @sethsaperstein-lyft in #296
• [STRMCMP-1659] k8s 1.24 upgrade networking fix by @sethsaperstein-lyft in #297
• Change app status to rollback on job start timeout by @avneet15 in #298
• Headless Service by @sethsaperstein-lyft in #299
• Add support for SchedulerName for Flink pods by @anandswaminathan in #300

New Contributors

• @danielmmetz made their first contribution in #259
• @sethsaperstein-lyft made their first contribution in #280
• @leoluoInSea made their first contribution in #285
• @avneet15 made their first contribution in #298

Full Changelog: v0.6.0...v0.7.0

What's Changed

• More doc detail for maxCheckpointRestoreAgeSeconds by @haltwise in #200
• [STRMCMP-944] Add proper validation for the sysctls field in securityContext by @mwylde in #204
• Add InPlace scale mode by @mwylde in #209
• Add Zillow to the Who is Using page by @kelly-sm in #210
• Minor/cosmetics changes on documentation by @rjbordon in #208
• Remove old flink-app-hash selector from generic service by @mwylde in #211
• Adapt to Flink v1.11 memory configuration by @alberto-quesada-scopely in #207
• Increase job submission timeout and fast fail when it occurs by @mwylde in #217
• [STRMCMP-1133] Fix for handling nil deployments by @premsantosh in #219
• Update CODEOWNERS by @mwylde in #224
• Fix regression in handling of default heap memory config by @mwylde in #223
• Introduce affinity field by @gydtaai in #233
• Add GitHub actions instant-failure by @keith in #241

New Contributors

• @haltwise made their first contribution in #200
• @rjbordon made their first contribution in #208
• @alberto-quesada-scopely made their first contribution in #207
• @gydtaai made their first contribution in #233
• @keith made their first contribution in #241

Full Changelog: v0.5.0...v0.6.0

This release includes a number of feature additions and a couple of bug fixes. With this release, the operator now supports zero-downtime deployments by enabling a BlueGreen deployment mode. This is an opt-in feature and the modified state machine for BlueGreen deployments can be found here. Other feature enhancements include a SavepointDisabled mode to opt-in to updates without savepoints and the ability to set pod tolerations and security context in the CRD.

Thanks to Kelly Smith, Anand Swaminathan, Micah Wylde, Ravi Magham, Lakshmi Rao and Nasrudin Bin Salim for contributing to this release.

• acb407e Updates for release v0.5.0
• 6f6a2eb Fix status update post-teardown (#194)
• 9fe7fc1 Implement DualRunning state to enable blue-green deploys (#186)
• 4c377a3 Setup for blue/green deployment mode (#180)
• 8dfdef5 Allow disabling Savepoint during updates (#184)
• 2404b56 Update documentation and crd validation for pod tolerations (#182)
• e191024 Add support for setting pod tolerations (#181)
• c3c2663 [STRMCMP-825] Improve error messaging when moving to DeployFailed in a non-recoverable state (#171)
• be521f6 [STRMCMP-844] Fix rollback when SubmittingJob times out (#172)
• ec67117 Add Cartrack into who-is-using (#169)
• 4142437 [STRMCMP-752] Support recovering from externalized checkpoint even when JM is down (#162)
• e85d47d Add status subresource permission for Operator (#163)
• f8b0a12 Enabling PodSecurityContext to be configured for JM and TM pod templates (#158)

This release includes a number of bug fixes and backward incompatible changes. Includes change to enable the status subresource for the FlinkApplication CRD and way SavepointInfo is updated by the operator.

Please make sure to update the CRD and this update should not be deployed to a cluster where there are active flinkapplication updates occurring — i.e., all flinkapplications should be in a Running or DeployFailed state.

Thanks to Micah Wylde, Lakshmi Rao, Thomas Weise, Chaoran Yu, Temo Ojeda, Marcelo Da Cruz Pinto and Prem Santosh for contributing to this release.

Changes in this release:

• e9054a4 Upgrade version to 0.4.0 (#157)
• bb8834d Add Beam example (#134)
• 7d7379b Properly initialize http client to fix multiple submission issue (#153)
• 7fc2230 [STRMCMP-437] Making the checkpoint restoration age configurable (#151)
• 2a5baaf Provide operator configuration via FLINK_PROPERTIES fixes #135 (#149)
• 8460804 Add missing dot in readme (#147)
• a42556a Gate statemachine progress on taskmanager availability (#144)
• 11eec01 Various minor fixes (#145)
• 437a1ad Add error log for ForceCancel job (#143)
• 4edffc5 Fixes to ensure Delete phase is always handled and prevent deploys from being stuck in a Savepoi
  nting phase (#142)
• c2158e7 Update wordcount example to use docker-flink image #135 (#136)
• c9d77b8 Fix the wordcount example Flink config (#130)
• ea63c34 Update quickstart guide to point to consistent versions of cluster resources (#133)
• f499e7f [STRMCMP-675] Enable status subresource in CRD (#126)
• 5955172 Bumping to version 0.3.0 (#120)

This release includes a number of bug fixes. It also includes a breaking change in how Flink heap configurations are rendered. Previously this config was rendered incorrectly, which caused Flink to ignore it. This change will also cause all Flink applications to be redeployed when the operator is updated.

Thanks to Anand Swaminathan, Chaoran Yu, Lakshmi Rao, Marcelo Da Cruz Pinto, and Jonathan Miles for contributing to this release.

Changes in this release:

• fb6376a Update version to 0.3 (#118)
• a45dc4c [STRMCMP-639] Trigger immediate rollback when job submission fails (#117)
• 8630a9b Fix example Dockerfile (#116)
• 01eb3b8 Use a separate error metric for update conflicts (#113)
• 1612320 Update slack link (#114)
• 93708fc Update Slack invite link (#112)
• 2cff272 [STRMCMP-661] Fix bug that could lead to multiple job submission (#111)
• 77a7532 [#97] [STRMCMP-648] [BREAKING] Fix rendering of flink memory configs (#106)
• 674f2ca [STRMCMP-616] Improvements to integration test stability (#110)
• f2fd7a0 Updated Go to 1.13, controller-runtime to 0.2.2, added Lightbend to who-is-using (#105)
• 013789b Keep config names constant across (#101)
• 3ef97cb McAfee Update who-is-using.md (#103)
• 41def1a Upgrade to Controller Runtime v0.2.0 (#99)
• b1607d0 Failures during job startup should trigger immediate rollback (#100)
• b427995 Add The Trade Desk to list of users (#98)
• 7117482 Add community info to doc (#95)
• ed34700 Move FlinkApplication error (#94)
• e648d9c Make Ingress creation optional (#90)
• ba1533b Add Web UI information to status (#81)
• 7b52ad3 [STRMCMP-616] Use error retrying system for savepoint errors during deletion (#87)
• 2d8a52e Added an optional config in the CR to specify Flink application service account (#83)
• 94725bf Log events for all rollback situations (#86)
• 126caaa Add slack badge (#84)
• 2ca1af0 Update the word count image tag link
• 8fed5e3 [STRMCMP-610] Fix flink config rendering (#75)

373cb21 [Backward Incompatible] Add env variables for Flink managers and fix for JM HA (#45)
9053e4f Fix unit test from merge to master
bb7c8dd Enable allowNonRestoredState in flink CRD (#43)
1a62f03 [STRMCMP-558] Event improvements (#44)
8d847d7 Improve staleness logic by improving error handling and retrying a subset of errors (#32)
4297702 Find the correct deployments for readiness and status checks (#42)
7fda241 Add more metrics to Flinkk8soperator (#41)
9a95850 Initialize Klog explicitly using InitFlags (#40)
f920664 [Backward Incompatible] Upgrade controller runtime and Multi Namespace support for Flink operator (#39)
9aaa992 Add support for non restored state on job submission (#38)
cde4f38 Update logic around number and healthy task managers in status (#37)
6d92339 Use EventRecorder instead of emitting events manually (#36)

Changelog

6dd5f55 Adding openAPIV3Schema Validation for the Flink application custom resource (#33)
2f66087 [STRMCMP-544] Allow FlinkApplications to be deleted when job submission fails (#30)
1456182 [STRMCMP-509] Make old cluster clean-up idempotent (#27)
93f0e01 Fix Resty logging, timeout and retry configuration (#28)
9e35f64 Made sample app run on OpenShift (#26)
2e53557 Added doc on mounting volumes (#25)
d9dd430 Handle invalid phases in the Flink state machine (#23)
1616e73 Add node selectors to job and task config (#7)
8bfdcff Upgrade to latest version of FlyteStdlib (#21)
b8ff17d Minor fixes and improvements (#17)
d1d1176 Log operator config on startup (#11)
75f04e5 Add update/delete verb for service (#8)
5dbd1bd Added finalizer permissions (#12)
d3ff566 Add additional printer columns (#16)
390fa96 Fix the travis condition to make docker hub push on merge to master (#15)
f3c05db Add Lakshmi as codeowner (#14)
5aa8a13 Optimize image pushes and integration tests (#5)