17:20 <blackboxsw_> #startmeeting cloud-init bi-weekly office-hours
17:20 <meetingology> Meeting started at 17:20:28 UTC.  The chair is blackboxsw_.  Information about MeetBot at https://wiki.ubuntu.com/meetingology
17:20 <meetingology> Available commands: action, commands, idea, info, link, nick
17:22 <rharper> mal41: right, udev rules do the mapping (for azure, not sure if any other clouds are doing udev rule mapping as well); I guess I don't understand the use-case you have;
17:23 <blackboxsw_> community-notice: Hi cloud-init folks. Bi-weekly office hours meeting has begun. We keep this time slot available to encourage designated periods of time where at least one  upstream cloud-init committer is available with eyes on the channel for any discussions about bugs, features, feature-requests and general support. Any cloud-init discussions are welcomed..
17:23 <blackboxsw_> #chair rharper
17:23 <meetingology> Current chairs: blackboxsw_, rharper
17:24 <blackboxsw_> heya.... since you are around too :)
17:25 <rharper> =)
17:25 <blackboxsw_> the goal of the meeting time is an informal "meeting" where any questions and discussions can potentially gain broader audience... and anyone blocked can be unblocked whether it's for PR reviews, feature-work or general bug discussion.
17:26 <blackboxsw_> anyhow. I've said my piece, we generally also put up high-level discourse post for out-of-band readers that might want to keep up to date on recent commits/changes in cloud-init
17:26 <blackboxsw_> #link https://discourse.ubuntu.com/t/cloud-init-status-07-13-2021/23257
17:27 <mal41> rharper Aside from being able to change a device in a single place, it would also be possible to define things like disk_setup and mounts once using aliases, and then have device_aliases overridden by a more instance specific secondary config. In my that would certainly reduce the amount of duplication between otherwise very similar configs which
17:27 <mal41> is one of the reasons I started looking at it.
17:27 <mal41> *in my case
17:27 <blackboxsw_> note: upcoming upstream release is coming in august and we hope to get post-boot hotplug support into that release.
17:35 <blackboxsw_> rharper: mal41, so that I understand the discussion, is the "bug" that you suggest filing to remove/correct docs on  https://github.com/canonical/cloud-init/blob/main/cloudinit/config/cc_disk_setup.py#L19-L23  to avoid claiming support for device_aliases?
17:41 <mal41> I would prefer that the bug be to implement that feature as documented - if I understand correctly it seems achievable via couple of small modifications to cc_disk_detup and cc_mounts.
17:41 <blackboxsw_> paride dunno what happened with Travis CI, but it appears "unstuck" after I ran the CRON CI run on main and is now building the PR941 https://travis-ci.com/github/canonical/cloud-init/builds/232639556. So we might expect a run from your PR shortly and we can merge it
17:42 <mal41> But if it needs to be a low hanging fruit thing, then yes, removing the actively misleading documentation would be good until such time that a PR is ready and acceptable (of which part could be to revert any doc removal actioned in the interim).
17:43 <blackboxsw_> mal41: I think it sounds like a reasonable feature request bug if you can state the use-case you are looking for. I don't think we would be averse to adding such a feature, just a question of priortizing it. We certainly can help shepherd in PRs that interested folks submit on this ;)
17:44 <blackboxsw_> so either way, additional context on your use case in a but would help establish some level of priority and also give us something to point at as others may be looking for the same feature from userdata.
17:46 <rharper> mal41: right, I understand the usefulness of the aliases, I meant more practically, how does one on a given platform actually predict (or configure) that a specific block device (/dev/sda) is the disk that you really want to call 'my-data-drive' as an alias ;  and note that /dev/sdX and friends are unstable across reboots/kernels/platform changes (some providers may be stable, but it's not guaranteed);
17:50 <mal41> Hmm, I don't think this solves (and doesn't seek to solve) that issue - {"device_aliases": {"abc": "/dev/sdb"}, "disk_setup": {"abc": { ...cfg ...} } would suffer the only the same issue as with { "disk_setup": {"/dev/sdb": { ...cfg ...} } currently.
18:15 <mal41> blackboxsw_ Apologies, I missed your question; the use it would be in my case would be better readability in the config, a single point of definition for devices referenced in disk_setup and mounts, and following from that the ability to override the aliases from a secondary config without needing to completely redefine those sections. The primary
18:15 <mal41> case for the latter is when moving long-lived disks and instances between hardware in the cloud. i.e. Modify secondary config on OS disk ahead of time. Stop the instance, dettach/reattach disks to match new expectation in secondary config. Start the instance and have cloud-init understand the new disk devices and treat them appropriately without
18:15 <mal41> the need to change the config that was baked into the instance at original provisioning time.
18:17 <mal41> I'm certainly not oblivious to that probably being a very rare case, it's just one of those born of the stuff landing on my desk. :P
18:35 <mal41> I slapped together this very ugly draft just to confirm I'm on the right lines, if you have a moment to give it a glance. It needs tests, real world testing, and clean up, but hopefully it's a not-insane start, ;)  https://github.com/canonical/cloud-init/compare/main...mal:device-aliases?expand=1
18:35 <blackboxsw_> heh, understood the "stuff on my desk".  Yeah, I'm trying to wrap my head around how the detatch/reattach migration migration would work from cloud-nit perspective.
18:36 <blackboxsw_> I'm trying to understand what you mean by ". Modify secondary config on OS disk ahead of time" llike supplementary #cloud-config userdata?
18:39 <mal41> Yeah, so at the moment in this scenario I write out a smaller config affecting those areas I need to update, then modify the systemd service file to add the "--file /etc/cloud/override.yml" flag to the init call. Then when the system comes back after being stopped, it sees the instance ID has changed (as it's now on different cloud hardware) and
18:39 <mal41> runs cloud-init but with the modifications injected by the on disk config.
18:40 <blackboxsw_> hrm, so both "source storage machine" with orig long-lived disks and "target new storage target machine" would have been deployed with the same storage mount aliases and config expectations, and the migration process would be to detach orig disks for source machine, attach to target machine and launch target machine where the config is properly adapted and mount .
18:43 <mal41> In a sane world yes, but when you're handed VMs that have persistent data on their OS disk which can't be mounted on the new instance, then you start seeing the lovely idea of using a new properly configured instance slowly drift out of the window. :(
18:43 <mal41> "Can we just move the data?" "Not at this time." ¬_____¬
18:56 <blackboxsw_> please correct me on my understanding (now that I've looked over your diff) ....
18:56 <blackboxsw_> mal41: ok, so simply, for "migration reasons" you want to be able to simply support the documented device_aliases config dict as variable replacement values across both cc_disk_setup and cc_mount config modules to avoid repetition in user-data definitions because in your case you generally know what those alias to device path mappings will be on the "new" system
19:11 <blackboxsw_> I put that question simply so you can correct me; but I was wondering if YAML anchors and aliases themselves would work for the case of defining repeated path cases within the mounts module in the absence of this feature.    That said, looking over your diff I get the context a bit and your desire to use device_aliases config keys in both mounts and cc_disk_setup.
19:12 <blackboxsw_> In both of those patch cases, I'd prefer seeing something that'd fallback to the originally behavior if the key wasn't present in device_aliases.
19:14 <blackboxsw_> what you have written for disk_setup seems to do that.... first a lookup in device_aliases cfg (if unset we can just make that an empty dict {}) and we can avoid the if/else handling.
19:17 <blackboxsw_> but.... yeah, a good example #cloud-config that you expect you'd use would help clarify your desired flow.
19:19 <blackboxsw_> ok, let's wrap up office hours for today. Thanks all for the discussions, we can continue device_aliases discussions as needed or as a PR or bug is created for ongoing discusssion. As always, drop in with any questions you might have. Thanks again
19:19 <blackboxsw_> #endmeeting