Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

org-mode is broken after restarting emacs on latest develop when emacs < 29 #15896

Open
nixmaniack opened this issue Jan 19, 2023 · 92 comments
Open

Comments

@nixmaniack
Copy link
Contributor

Description :octocat:

org-mode is broken. Opening an org file doesn't load the org-mode.

I tried removing all org packages, restart emacs to let it install all the packages and then open an org file which works for that session. If you quit and restart it throws (invalid-function org-assert-version) error.

Reproduction guide 🪲

  • Delete org-9.6.1 and org-contrib-0.4.1 from ~/.emacs.d/elpa/28.2/develop.
  • Start emacs.
  • Let it install packages.
  • Open any org file, this loads org-mode file for the session.
  • Restart emacs.
  • Open any org file, it throws error.

Observed behaviour: 👀 💔
Opening an org file throws following error.

File mode specification error: (invalid-function org-assert-version)

Expected behaviour: ❤️ 😄
Opens file and applies org-mode and related layer configuration.

System Info 💻

  • OS: gnu/linux
  • Emacs: 28.2.50
  • Spacemacs: 0.999.0
  • Spacemacs branch: develop (rev. d08822c)
  • Graphic display: t
  • Running in daemon: nil
  • Distribution: spacemacs
  • Editing style: hybrid
  • Completion: ivy
  • Layers:
(auto-completion emacs-lisp git ivy lsp markdown multiple-cursors org
                 (shell :variables shell-default-height 30 shell-default-position 'bottom shell-default-shell 'vterm)
                 syntax-checking version-control treemacs themes-megapack)
  • System configuration features: ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ IMAGEMAGICK JPEG JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB
@lebensterben
Copy link
Collaborator

cc @smile13241324

@lebensterben
Copy link
Collaborator

Confirmed. At first my emacs was working fine. But after deleting the two packages mentioned above, Emacs won't start properly. (spacemac-buffer won't show up since it requires org with my settings)

@lebensterben
Copy link
Collaborator

lebensterben commented Jan 19, 2023

Inspired by https://irreal.org/blog/?p=10999

To fix it:

  1. cd ~/.emacs.d; rm -rf elpa/28.2/develop/org-9*

  2. start emacs with emacs -Q and evaluate (with C-j)

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir
      (expand-file-name "develop"
                        (expand-file-name emacs-version
                                          (expand-file-name "elpa"
                                                            user-emacs-directory))))
(package-refresh-contents)
(package-install 'org)
  1. Restart Emacs normally.

@nixmaniack
Copy link
Contributor Author

@lebensterben I followed your workaround but looks it didn't work for me. I did rm -rf the org packages but the last line to install org throws this message.

"‘org’ is already installed"

I just followed your exact instruction and haven't dug into the blog post. I will check what's going on later. One thing I noticed is that, emacs-version is reported as 28.2.50 which makes the path elpa/28.2.50/develop but all the other packages spacemacs has installed are in elpa/28.2/develop.

@lebensterben
Copy link
Collaborator

make sure you removed all org-9.6* in your elpa directory.

@smile13241324
Copy link
Collaborator

Strange, this seems only to happen on emacs 28, not on emacs 29 and it seems to be independent from what is entered into the spacemacs buffer, setting it to show no content at all (no lists no agenda) has the same effect.

When I am trying to require org in my dotfile I am seeing the following:

Debugger entered--Lisp error: (invalid-function org-assert-version)
  org-assert-version()
  byte-code("\300\301!\210\302 \210\300\303!\210\300\304!\207" [require org-macs org-assert-version cl-lib oc] 2)
  #<subr require>(org-keys)
  apply(#<subr require> org-keys)
  require(org-keys)
  eval-buffer(#<buffer  *load*> nil "/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." nil t)  ; Reading at buffer position 3557
  load-with-code-conversion("/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." "/home/smile13241324/.emacs.d/elpa/28.2/develop/org..." nil t)
  #<subr require>(org)
  apply(#<subr require> org)
  require(org)
  (progn (require 'org))
  eval((progn (require 'org)) t)
  elisp--eval-last-sexp(nil)
  #f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>)(nil)
  #f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>)()
  eval-sexp-fu-flash-doit-simple(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>))
  eval-sexp-fu-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>))
  esf-flash-doit(#f(compiled-function (&rest _it) #<bytecode 0x1fff02a74872>) #f(compiled-function (&rest args2) #<bytecode -0x1133d5a2cfbd057f>) #f(compiled-function (&rest args2) #<bytecode -0x112bf59b0182057f>) #f(compiled-function (&rest args2) #<bytecode 0x153e757edc0c5a87>))
  ad-Advice-eval-last-sexp(#f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>) nil)
  apply(ad-Advice-eval-last-sexp #f(compiled-function (eval-last-sexp-arg-internal) "Evaluate sexp before point; print value in the echo area.\nInteractively, with a non `-' prefix argument, print output into\ncurrent buffer.\n\nThis commands handles `defvar', `defcustom' and `defface' the\nsame way that `eval-defun' does.  See the doc string of that\nfunction for details.\n\nNormally, this function truncates long output according to the\nvalue of the variables `eval-expression-print-length' and\n`eval-expression-print-level'.  With a prefix argument of zero,\nhowever, there is no such truncation.\nInteger values are printed in several formats (decimal, octal,\nand hexadecimal).  When the prefix argument is -1 or the value\ndoesn't exceed `eval-expression-print-maximum-character', an\ninteger value is also printed as a character of that codepoint.\n\nIf `eval-expression-debug-on-error' is non-nil, which is the default,\nthis command arranges for all errors to enter the debugger." (interactive "P") #<bytecode -0x45c973c4e971b99>) nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  call-interactively(eval-last-sexp nil nil)
  command-execute(eval-last-sexp)

I think we need more investigation into this first.

@nixmaniack
Copy link
Contributor Author

I had to explicitly set the package-user-dir to a value that normal spacemacs sets up and I was able to install org with the instructions provided. For now, the workaround seems to work. Thanks @lebensterben!

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir "/home/localuser/.emacs.d/elpa/28.2/develop/")
(package-refresh-contents)
(package-install 'org)

@hny-gd
Copy link

hny-gd commented Feb 17, 2023

For me running

cd ~/.emacs.d; rm -rf elpa/develop/org-9* (note develop instead of 28.2 as @lebensterben wrote above)

and then starting Emacs was enough.

Spacemacs then reinstalled org and everything was fine.

@emacs18
Copy link
Contributor

emacs18 commented Feb 18, 2023

There has been many problems like this over the years due to org mode. Emacs comes with org mode, but almost always folks need to install a newer version. This newer version is also needed by other packages as well. That is why I make sure that org package is installed very early on in early-init.el. I used to do this when I used package.el years ago. Since I've been using straight.el for few years, I have (straight-use-package 'org) very early on in my early-init.el file to assure that org package is installed before even getting to most spacemacs initialization code.

@RidaAyed
Copy link

I had to explicitly set the package-user-dir to a value that normal spacemacs sets up and I was able to install org with the instructions provided. For now, the workaround seems to work. Thanks @lebensterben!

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir "~/.emacs.d/elpa/28.2/develop/")
(package-refresh-contents)
(package-install 'org)

Note that in my case, alongside the existing package directory (e.g. ~/.emacs.d/elpa/develop) the upper snippet downloads and installs all packages into the package-user-dir and finally reports: ‘org’ is already installed

make sure you removed all org-9.6* in your elpa directory.

M-x org-version still reports Org mode version 9.5.5 (release_9.5.5 @ /usr/share/emacs/28.2/lisp/org/)

@lebensterben
Copy link
Collaborator

@RidaAyed

#15896 (comment)

have you first removed org from elpa directory?

@emacs18
Copy link
Contributor

emacs18 commented Feb 23, 2023

I don't think org package is in melpa. You need to add https://elpa.gnu.org/packages/. Following is how spacemacs sets up packages.

(defun configuration-layer/create-elpa-repository (name output-dir)
  "Create an ELPA repository containing all packages supported by Spacemacs."
  (configuration-layer/make-all-packages 'no-discover)
  (let (package-archive-contents
        (package-archives '(("melpa" . "https://melpa.org/packages/")
                            ("gnu"   . "https://elpa.gnu.org/packages/")
                            ("nongnu" . "https://elpa.nongnu.org/nongnu/"))))

@lebensterben
Copy link
Collaborator

if one followed my instructions melpa is added.

@notestaff
Copy link

FWIW, removing the byte-compiled *.elc files in the installed elpa org directory seems to fix the org-assert-function issue for me (but I use standard emacs not spacemacs).

@emacs18
Copy link
Contributor

emacs18 commented Feb 24, 2023

Corrupt byte compiled files could result if you installed org package after already having loaded built-in org.el.
This is why one must make sure that org.el is not loaded when you install org package!
This is why I have code in early-init.el (rather than init.el) to not only setup package archives, but also to install org packages.
Because this is done so early on during emacs start-up, there is no possibility of org.el being loaded when org package is installed.

I think this is the most common problem with org mode the folks have had over the years.

@lebensterben
Copy link
Collaborator

@emacs18 will there be any impact on loading time?

@emacs18
Copy link
Contributor

emacs18 commented Feb 24, 2023

I don't think so, but I am not sure. Even if startup time is longer, I could tolerate it if that prevents hard to debug errors.

@tko
Copy link
Contributor

tko commented Feb 24, 2023

I think this might get fixed by deleting all of elpa/ -- no guarantees though. (Two computers, on one I deleted everything for unrelated reasons and it works ok, on the other I've just kept upgrading things and hit this same error. Was planning to delete elpa when I have the time.)

@sir4ur0n
Copy link
Contributor

@tko I just tested and it doesn't solve the problem for me 😞

@RidaAyed
Copy link

RidaAyed commented Feb 27, 2023

Inspired by https://irreal.org/blog/?p=10999

To fix it:

1. `cd ~/.emacs.d; rm -rf elpa/28.2/develop/org-9*`

2. start emacs with `emacs -Q` and evaluate (with `C-j`)
(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/"))
(setq package-user-dir
      (expand-file-name "develop"
                        (expand-file-name emacs-version
                                          (expand-file-name "elpa"
                                                            user-emacs-directory))))
(package-refresh-contents)
(package-install 'org)
3. Restart Emacs normally.

@lebensterben

Running your snippet returns "‘org’ is already installed"

Yes, even deleted the complete elpa folder. Tried even deleting /usr/share/emacs/28.2/lisp/org/ to no avail. After reinstalling emacs yay emacs > reinstall I'm back to "‘org’ is already installed"

@lebensterben ..and of course: Thank you very much for your time 🙏

@lebensterben
Copy link
Collaborator

@RidaAyed

what's your emacs version

@RidaAyed
Copy link

@lebensterben

System Info 💻

  • OS: gnu/linux
  • Emacs: 28.2
  • Spacemacs: 0.999.0
  • Spacemacs branch: develop (rev. 8645f21cb)
  • Graphic display: t
  • Running in daemon: nil
  • Distribution: spacemacs
  • Editing style: vim
  • Completion: ivy

@emacs18
Copy link
Contributor

emacs18 commented Feb 27, 2023

@RidaAyed

Can you explain how hour setup can install org package at all?

You set package-archives to include only melpa. As far as I can tell, melpa does not provide org package! Thus I don't see how you could possibly be installing org package when package-archives does not include https://elpa.gnu.org/packages/ as well.

@emacs18
Copy link
Contributor

emacs18 commented Feb 27, 2023

Sorry. I just noticed that you are using add-to-list. So you are probably relying on default value of package-archive to already have elpa.gnu.org.

@RidaAyed
Copy link

@emacs18 I was referring to to the snippet from @lebensterben

Otherwise I'm relying on the standard spacemacs layers

dotspacemacs-configuration-layers
'(
  (org :variables
       org-enable-hugo-support t
       org-enable-valign nil
       )
  )

@nixmaniack
Copy link
Contributor Author

@RidaAyed Your snippet sets the value of the package-user-dir to a different value than what spacemacs computes. I had similar issues when I was debugging hence I hardcoded the value for the workaround. You can check how spacemacs computes the value here but in short it strips the patch version from the version string. So even though the workaround downloads the fixed org version, it's never used by Spacemacs.

For clarity, on my machine, Spacemacs computes package-user-dir as /home/localuser/.emacs.d/elpa/28.2/develop/, and your snippet computes it as /home/localuser/.emacs.d/elpa/28.2.50/develop.

@bryce-carson
Copy link
Contributor

It's definitely a strange issue. Installing Org from ELPA using the package interface after setting package-user-dir correctly did not fix the issue, nor did recompiling at that stage. Until I installed from a TAR file, recompiled, reinstalled the other packages, and then finally restarted, the issue did not go away.

I do not think, installing directly from tar is relevant.

invalid function: org-assert-version

means that Org was compiled when an old (likely built-in) version of Org is already loaded (through dependency, when opening an Org file, because help autocompletion offered a function or variable from Org, etc.). batch-byte-compile and byte-recompile-directory functions can not help to recover from such state. Remove Org .elc files manually or delete org package.

The following commit from the emacs-29 branch fixes update of loaded package by package.el: https://git.savannah.gnu.org/cgit/emacs.git/commit/?id=9dfd945a2c2 https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49708 "Fix byte compilation of package built-ins"

A workaround for emacs < 29 may be unzipping /usr/share/emacs/<VERSION>/lisp/org/*.el.gz files. It should fix loading of .el files from the update over earlier loaded .elc files. If you do not like such trick, ensure that (featurep 'org) is evaluated to nil when your are going to install new Org version from ELPA, either from list-packages or from a file.

If that also works, that is good for a select few users. The current work-around, however, does not rely on super-user privileges for modifying system files; some distributions may detect changes in system files associated with packages and cause issues with system package managers. I'd prefer to avoid that.

Regardless, isn't the issue with the byte-compiled files in that directory, not the compressed source files? If I knew more about how Emacs and package.el worked I could tinker with a fool-proof, portable solution, but as I mentioned previously (no longer visible due to comment edits) it's not worth the time investment. The current work-around is portable enough that if a user wants a work-around, if they've found their way here, they are more likely to have success running the simple commands until the errors disappear.

Keeping the work-around as simple as possible, even if it has more steps or takes another minute, is favourable because it introduces less headache. Reinstalling multiple packages, granted, may cause an issue in an exotic package (but given it's being reinstalled because of dependency issues there was kludge already), but in general it should be safe and portable.

@maxnikulin
Copy link

invalid function: org-assert-version

I forgot to mention that the "invalid function" means that org-assert-version is defined as a macro in current session, so new org-macs.el file is loaded, but during compilation org-assert-version was undefined because of old org-macs.el from built-in Org, so instead of macro expansion, a function call is added to the .elc file caused the error. Do not confuse it with

Symbol’s function definition is void: org-assert-version

that means an attempt to load a new incorrectly compiled .elc file while old org-macs is loaded. It means that for some reason built-in Org is loaded before the directory with new Org is added to `load-path'.

Regardless, isn't the issue with the byte-compiled files in that directory, not the compressed source files?

.elc files in ~/emacs.d/elpa/org-*/ are incorrectly compiled because of buggy package.el during installation of the ELPA package did not load updated .el files over built-in /usr/share/emacs/27.1/lisp/org/*.elc before compilation. The same package.el code is able to load new .el files when there are .el (not .el.gz) files in the same /usr/share directory and they are regular files, not symliks like in the elpa-org deb package.

Keeping the work-around as simple as possible, even if it has more steps or takes another minute, is favourable because it introduces less headache.

I do not see a reason to require installing namely from tar file. Either there are more bugs in package.el (perhaps not fixed yet in the development branch) or it is not necessary.

@maxnikulin
Copy link

Does spacemacs have some specifics that makes the following workarounds unusable?

  1. Install from a clean Emacs session
emacs -q
M-x package-delete RET org RET
M-x list-packages RET / n org RET

select Org, i x to install. Of course, the delete step is not required if the Org ELPA package has not been installed yet.

  1. Alternatively remove Org .elc files and compile them (ignoring init file) from command line
rm -v ~/.emacs.d/elpa/org-[0-9]*/*.elc
emacs -q --batch -L ~/.emacs.d/elpa/org-[0-9]*/ -f batch-byte-compile ~/.emacs.d/elpa/org-[0-9]*/*.el

Do not try byte-recompile-directory instead, it may load .elc files incorrectly compiled by package-install.

@smile13241324
Copy link
Collaborator

Nope there is nothing hindering users to use this workaround except that the issue may reappear each time the org package is updated :(.

To be honest I do not see much we can do to work around this problem, in principle its a package.el bug which has been fixed there already, and will not appear in emacs 29.1 users should consider upgrading rather then doing all these workarounds in my humble opinion.

@bryce-carson
Copy link
Contributor

Nope there is nothing hindering users to use this workaround except that the issue may reappear each time the org package is updated :(.

To be honest I do not see much we can do to work around this problem, in principle its a package.el bug which has been fixed there already, and will not appear in emacs 29.1 users should consider upgrading rather then doing all these workarounds in my humble opinion.

Indeed. I'd say once Ubuntu and Fedora both ship 29.1 in the official repo, Spacemacs should no longer officially support ealier versions of Emacs.

Perhaps we should consider advising users install Guix and then they'd get access to 29.1 immediately, rather than waiting for other distros to ship different builds of the same upstream tag of Emacs, we'd rely on one, reproducible build?

@bryce-carson
Copy link
Contributor

@smile13241324 For now, could you pin this issue, Maxi?, if you're able? It's relevant enough that it should have more visibility.

@smile13241324 smile13241324 pinned this issue Jun 11, 2023
@smile13241324
Copy link
Collaborator

@bryce-carson, pinned. Yes indeed once 29.1 is officially released we should set the minimum version to 29.1 at least until there is an Ubuntu and Debian build available for it.

@vatrat
Copy link

vatrat commented Jun 11, 2023

Just chiming in, once/if we are able to find an automatic workaround for this issue, we should implement it. Otherwise, add a check in spacemacs to abort startup and show a message warning that the current version is less than 29.1 with a link to this issue. This way, it would prevent the bug from making spacemacs unusable until the user acknowledges the issue. I see this as a more useful fix than setting a minimum version, as it's going to be quite a while before 29.x is propagated to every distribution and every user updates. In the meantime, what happens when someone goes to open spacemacs after a while and it unexpectedly bricks itself? Dropping support for previous versions only introduces more support headaches, as people will continue to report this bug for as long as it occurs in the wild. We have access to elisp and version checks, and I think we should use them. This is an unfortunate situation that we shouldn't have to deal with, but let's make the most of it.

I guess that I'm just opposed to focusing on popular distributions because I started using spacemacs on a chromebook in a free VM and all other cursed manner of setups. If we want the latest version to be a hard requirement, we need to direct new users on how to install it on any platform, even if that's just dropping a link to emacs' installation guide.

@bryce-carson
Copy link
Contributor

Just chiming in, once/if we are able to find an automatic workaround for this issue, we should implement it. Otherwise, add a check in spacemacs to abort startup and show a message warning that the current version is less than 29.1 with a link to this issue. This way, it would prevent the bug from making spacemacs unusable until the user acknowledges the issue. I see this as a more useful fix than setting a minimum version, as it's going to be quite a while before 29.x is propagated to every distribution and every user updates. In the meantime, what happens when someone goes to open spacemacs after a while and it unexpectedly bricks itself? Dropping support for previous versions only introduces more support headaches, as people will continue to report this bug for as long as it occurs in the wild. We have access to elisp and version checks, and I think we should use them. This is an unfortunate situation that we shouldn't have to deal with, but let's make the most of it.

I guess that I'm just opposed to focusing on popular distributions because I started using spacemacs on a chromebook in a free VM and all other cursed manner of setups. If we want the latest version to be a hard requirement, we need to direct new users on how to install it on any platform, even if that's just dropping a link to emacs' installation guide.

The plan is to change the minimum supported version in Spacemacs itself, which after some minor refactoring, should refuse to initialize on ealier versions of Emacs. That's not a problem for any motivated user of an earlier Emacs version. Spacemacs is not currently bricked, and Spacemacs will not brick itself in any case. The problem is actually quite minor.

Closed issues can remain pinned, and documentation, guides, and READMEs serve better the purpose of communicating known issues to users.

The only real fix would be a labourious backport. The current work-around has a much lower development threshold; granted, a better and automatic solution would be preferred, but there are considerations to make.

First, should we prevent users from updating Org automatically? If so, we can hold the package by specifiying a version string for it in early-init.el:

(customize-set-variable 'package-load-list '(all (org "9.6.6")))

The consideration then becomes, "What version?" We might leave this up to users to decide, simply providing them the instruction so they only need to perform the current work-around procedure once, hold the package, and then forget about the issue.

An automated procedure is too delicate to spend our time on, I feel.

Another solution is to prevent Org from loading at all by setting the cdr of the list containing the symbol org to nil: (customize-set-variable 'package-load-list '(all (org nil))). This will prevent all versions of Org from being loaded at startup, and then Spacemacs might be able to initialize a reinstallation automatically, and only once before holding the package, or simply loading a freshly compiled version of Org or explicitly loading Emacs Lisp files only for Org, and never byte compiling the package.

There's multiple solutions, but I'm not sure which would be best. I think instructing users to hold the Org package at a desired version using the above form after following the current work-around procedure is easiest.

@vatrat
Copy link

vatrat commented Jun 11, 2023

@bryce-carson I agree with you. I'm thinking that refusing to load org may be the best option, as it preserves other functionality by quarantine, disabling org rather than trying to make spacemacs fix the issue. This way, people who don't use org are unaffected and if there's still demand for a fix, pre-29.1 users can work to explore other options. In my eyes, the main benefit of this solution is making spacemacs "just work" for people who don't understand the inner workings.

@bryce-carson
Copy link
Contributor

As a follow-up, rather than editing, when Spacemacs drops support for a version of Emacs it only means that if the reported issue is caused by something in that version, such as the present issue, we probably won't make an effort to provide a work-around.

This issue is unique because it's caused by package.el and interacts exotically with Org. Other packages aren't suffering this issue (yet). In the current supported versions of Emacs, 27.1 or 28.2, the question "Is it an issue with Org, or package.el? Which do we address?" can be asked.

As I said in the previous comment, the fixes to package.el in 29.1 could be back-ported, which is labourious indeed. It's also fidddlesome, apart from the labour needed. It could introduce more issues.

It might be easier to hold Org to the 9.5 series, especially for new installations, and if a user wishes to upgrade Org then the work-around and further package version hand-holding would be advised.

@vatrat
Copy link

vatrat commented Jun 11, 2023

Hmm... So keeping org at 9.5.x by default (if a version below 29.1 is detected) would work? If so, I agree, that's a better idea. Is it possible to limit this behavior to only pre-29.1 versions, so that new installs on 29.1 or above don't get stuck with an old org package?

@bryce-carson
Copy link
Contributor

bryce-carson commented Jun 11, 2023

It's trivial, actually. We'd simply add the following to early-init.el.

(when (version< emacs-version "29.1")
  (customize-set-variable 'package-load-list '(all (org "9.5.5"))))

I'm not sure how related to the present issue it is, the upstream package website formerly published its own ELPA archive for Org packages (see https://orgmode.org/elpa.html). Org 9.5 was the last package version to work with this (Org 9.5.5 is distributed from GNU ELPA directly), and Org ELPA itself might become deprecated and offlined in the future. While we wait... a year? for 29.1 to land in Ubuntu (Fedora will have it in 39 if Emacs 29.1 gets released ahead of schedule) we might find other issues caused by Org ELPA's past.

Org 9.5 will be the last release to be distributed on Org ELPA. After
9.5, we won't use Org ELPA for new releases. Past releases will still
be available, though maybe not indefinitely.

With the "trivial" addition to early-init, we need to ask whether this will causes issues in sites with more recent versions of Org already installed, and how that will interact with byte compilation on 28.2, the portable dumper on 27.1, etc.


Edit: clarified a clause.

Org 9.5 was the last package version to work with this (Org 9.5.5 is distributed from GNU ELPA directly)

@vatrat
Copy link

vatrat commented Jun 11, 2023

With the "trivial" addition to early-init, we need to ask whether this will causes issues in sites with more recent versions of Org already installed, and how that will interact with byte compilation on 28.2, the portable dumper on 27.1, etc.

Oh boy, I'm starting to see what you mean. In the meantime, applying that "trivial" fix to new installations seems safe, assuming spacemacs has a way of detecting that.

@bryce-carson
Copy link
Contributor

With the "trivial" addition to early-init, we need to ask whether this will causes issues in sites with more recent versions of Org already installed, and how that will interact with byte compilation on 28.2, the portable dumper on 27.1, etc.

Oh boy, I'm starting to see what you mean. In the meantime, applying that "trivial" fix to new installations seems safe, assuming spacemacs has a way of detecting that.

Indeed. That's why communication is easier. Spacemacs does not have as many active hackers as Doom Emacs does, and even fewer contributors understand it all.

The best way to ensure new installations include the change is to provide the code to users in the installation instructions. Feel free to submit a PR against the README if you want to get involved. Spacemacs needs more active contributors hacking on and proofing the documentation.

@vatrat
Copy link

vatrat commented Jun 12, 2023

Appreciate the response at all, I'll look at it when I'm done with my current work.

@bryce-carson
Copy link
Contributor

Appreciate the response at all, I'll look at it when I'm done with my current work.

Sounds good. I'm investigating the consequences of shipping the change by default (27.1 and such).

@maxnikulin
Copy link

In the current supported versions of Emacs, 27.1 or 28.2, the question "Is it an issue with Org, or package.el?

The primary issue with Org is that it is a large package with multiple files and macros are actively used. The latter is a challenge for Emacs byte compilation. Void/invalid function issues were not rare reports on the Org mailing list, and it is a reason for the https://orgmode.org/worg/org-faq.html#mixed-install note. The idea of org-assert-version was to make clear to users that something is wrong in their configuration. I do not think that at the moment when it was introduced, it was clear that package.el can not handle update of a loaded package.

So keeping org at 9.5.x by default (if a version below 29.1 is detected)

Wouldn't it better to keep the built-in version of Org?

@maxnikulin
Copy link

once/if we are able to find an automatic workaround for this issue, we should implement it.

Once invalid function: org-assert-version is detected, temporary suppress loading of .elc files (load-suffixes), reload org-macs.el, run byte-recompile-directory for Org.

@bryce-carson
Copy link
Contributor

In the current supported versions of Emacs, 27.1 or 28.2, the question "Is it an issue with Org, or package.el?

The primary issue with Org is that it is a large package with multiple files and macros are actively used. The latter is a challenge for Emacs byte compilation. Void/invalid function issues were not rare reports on the Org mailing list, and it is a reason for the https://orgmode.org/worg/org-faq.html#mixed-install note. The idea of org-assert-version was to make clear to users that something is wrong in their configuration. I do not think that at the moment when it was introduced, it was clear that package.el can not handle update of a loaded package.

So keeping org at 9.5.x by default (if a version below 29.1 is detected)

Wouldn't it better to keep the built-in version of Org?

That would be the purpose behind holding the package, to prefer the built-in package. On most platforms, that seems to be 9.5.5; for any platform that has a version less than 9.5, it's okay to uprade to 9.5.x, just not to 9.6.

For systems where 9.5.5 is distributed, it keeps the built-in, otherwise we allow upgrade to the newer version that doesn't produce the error.

That is only if we introduce this breaking change, however.

@wztdream
Copy link
Contributor

I encountered the problem while upgrading Spacemacs and was able to resolve it by deleting the old org files. However, I encountered some errors initially, which were resolved after restarting Spacemacs. Although I accidentally resolved the problem, I am concerned that it may resurface later. Therefore, I believe it is necessary to investigate the issue related with the old org files in more detail.

@allentiak
Copy link
Contributor

allentiak commented Jul 22, 2023

I've just bumped into this (ironically) after applying " Update htmlize \ Fix for the org version conflict." 6e5816b

Rolling back my packages to their 2023.07.12 version solved the problem.
That's why I always keep at least the last two updates in my "able-to-rollback" cache :)

@moritzschaefer
Copy link
Contributor

I tried @lebensterben and other ways to fix this but no luck. org-version returns the new correct version 9.6.7 but still I get (invalid-function org-assert-version)

@3dimaging
Copy link

@lebensterben I followed your workaround but looks it didn't work for me. I did rm -rf the org packages but the last line to install org throws this message.

"‘org’ is already installed"

I just followed your exact instruction and haven't dug into the blog post. I will check what's going on later. One thing I noticed is that, emacs-version is reported as 28.2.50 which makes the path elpa/28.2.50/develop but all the other packages spacemacs has installed are in elpa/28.2/develop.

It does not matter. I followed @lebensterben and also got the message, "‘org’ is already installed". I restart the emacs. The org model works fine.

@neverkas
Copy link

neverkas commented Oct 6, 2023

I did what said @lebensterben and worked until I restart spacemacs so I decided to install Emacs 29 because of someone here said that there was not problem and now it works.

@biocyberman
Copy link

In addition to installing Emacs 29.1, I had to put the following into (defun dotspacemacs/user-init () in .spacemacs

  (straight-use-package '(org :type built-in))

@smile13241324 smile13241324 changed the title org-mode is broken after restarting emacs on latest develop org-mode is broken after restarting emacs on latest develop when emacs < 29 Jan 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests