Vim documentation: autocmd
main help file*autocmd.txt* For Vim version 7.2. Last change: 2008 Jun 27 VIM REFERENCE MANUAL by Bram Moolenaar Automatic commands *autocommand* For a basic explanation, see section |40.3| in the user manual. 1. Introduction |autocmd-intro| 2. Defining autocommands |autocmd-define| 3. Removing autocommands |autocmd-remove| 4. Listing autocommands |autocmd-list| 5. Events |autocmd-events| 6. Patterns |autocmd-patterns| 7. Buffer-local autocommands |autocmd-buflocal| 8. Groups |autocmd-groups| 9. Executing autocommands |autocmd-execute| 10. Using autocommands |autocmd-use| 11. Disabling autocommands |autocmd-disable| {Vi does not have any of these commands} {only when the |+autocmd| feature has not been disabled at compile time}
1. Introduction *autocmd-intro* You can specify commands to be executed automatically when reading or writing a file, when entering or leaving a buffer or window, and when exiting Vim. For example, you can create an autocommand to set the 'cindent' option for files matching *.c. You can also use autocommands to implement advanced features, such as editing compressed files (see |gzip-example|). The usual place to put autocommands is in your .vimrc or .exrc file. *E203* *E204* *E143* WARNING: Using autocommands is very powerful, and may lead to unexpected side effects. Be careful not to destroy your text. - It's a good idea to do some testing on an expendable copy of a file first. For example: If you use autocommands to decompress a file when starting to edit it, make sure that the autocommands for compressing when writing work correctly. - Be prepared for an error halfway through (e.g., disk full). Vim will mostly be able to undo the changes to the buffer, but you may have to clean up the changes to other files by hand (e.g., compress a file that has been decompressed). - If the BufRead* events allow you to edit a compressed file, the FileRead* events should do the same (this makes recovery possible in some rare cases). It's a good idea to use the same autocommands for the File* and Buf* events when possible.
2. Defining autocommands *autocmd-define* Note: The ":autocmd" command cannot be followed by another command, since any '|' is considered part of the command. *:au* *:autocmd* :au[tocmd] [group] {event} {pat} [nested] {cmd} Add {cmd} to the list of commands that Vim will execute automatically on {event} for a file matching {pat}. Vim always adds the {cmd} after existing autocommands, so that the autocommands execute in the order in which they were given. See |autocmd-nested| for [nested]. The special pattern <buffer> or <buffer=N> defines a buffer-local autocommand. See |autocmd-buflocal|. Note that special characters (e.g., "%", "<cword>") in the ":autocmd" arguments are not expanded when the autocommand is defined. These will be expanded when the Event is recognized, and the {cmd} is executed. The only exception is that "<sfile>" is expanded when the autocmd is defined. Example: :au BufNewFile,BufRead *.html so <sfile>:h/html.vim Here Vim expands <sfile> to the name of the file containing this line. When your .vimrc file is sourced twice, the autocommands will appear twice. To avoid this, put this command in your .vimrc file, before defining autocommands: :autocmd! " Remove ALL autocommands for the current group. If you don't want to remove all autocommands, you can instead use a variable to ensure that Vim includes the autocommands only once: :if !exists("autocommands_loaded") : let autocommands_loaded = 1 : au ... :endif When the [group] argument is not given, Vim uses the current group (as defined with ":augroup"); otherwise, Vim uses the group defined with [group]. Note that [group] must have been defined before. You cannot define a new group with ":au group ..."; use ":augroup" for that. While testing autocommands, you might find the 'verbose' option to be useful: :set verbose=9 This setting makes Vim echo the autocommands as it executes them. When defining an autocommand in a script, it will be able to call functions local to the script and use mappings local to the script. When the event is triggered and the command executed, it will run in the context of the script it was defined in. This matters if |<SID>| is used in a command. When executing the commands, the message from one command overwrites a previous message. This is different from when executing the commands manually. Mostly the screen will not scroll up, thus there is no hit-enter prompt. When one command outputs two messages this can happen anyway.
3. Removing autocommands *autocmd-remove* :au[tocmd]! [group] {event} {pat} [nested] {cmd} Remove all autocommands associated with {event} and {pat}, and add the command {cmd}. See |autocmd-nested| for [nested]. :au[tocmd]! [group] {event} {pat} Remove all autocommands associated with {event} and {pat}. :au[tocmd]! [group] * {pat} Remove all autocommands associated with {pat} for all events. :au[tocmd]! [group] {event} Remove ALL autocommands for {event}. :au[tocmd]! [group] Remove ALL autocommands. When the [group] argument is not given, Vim uses the current group (as defined with ":augroup"); otherwise, Vim uses the group defined with [group].
4. Listing autocommands *autocmd-list* :au[tocmd] [group] {event} {pat} Show the autocommands associated with {event} and {pat}. :au[tocmd] [group] * {pat} Show the autocommands associated with {pat} for all events. :au[tocmd] [group] {event} Show all autocommands for {event}. :au[tocmd] [group] Show all autocommands. If you provide the [group] argument, Vim lists only the autocommands for [group]; otherwise, Vim lists the autocommands for ALL groups. Note that this argument behavior differs from that for defining and removing autocommands. In order to list buffer-local autocommands, use a pattern in the form <buffer> or <buffer=N>. See |autocmd-buflocal|. *:autocmd-verbose* When 'verbose' is non-zero, listing an autocommand will also display where it was last defined. Example: :verbose autocmd BufEnter FileExplorer BufEnter * call s:LocalBrowse(expand("<amatch>")) Last set from /usr/share/vim/vim-7.0/plugin/NetrwPlugin.vim See |:verbose-cmd| for more information.
5. Events *autocmd-events* *E215* *E216* You can specify a comma-separated list of event names. No white space can be used in this list. The command applies to all the events in the list. For READING FILES there are four kinds of events possible: BufNewFile starting to edit a non-existent file BufReadPre BufReadPost starting to edit an existing file FilterReadPre FilterReadPost read the temp file with filter output FileReadPre FileReadPost any other file read Vim uses only one of these four kinds when reading a file. The "Pre" and "Post" events are both triggered, before and after reading the file. Note that the autocommands for the *ReadPre events and all the Filter events are not allowed to change the current buffer (you will get an error message if this happens). This is to prevent the file to be read into the wrong buffer. Note that the 'modified' flag is reset AFTER executing the BufReadPost and BufNewFile autocommands. But when the 'modified' option was set by the autocommands, this doesn't happen. You can use the 'eventignore' option to ignore a number of events or all events. *autocommand-events* *{event}* Vim recognizes the following events. Vim ignores the case of event names (e.g., you can use "BUFread" or "bufread" instead of "BufRead"). First an overview by function with a short explanation. Then the list alphabetically with full explanations |autocmd-events-abc|. Name triggered by Reading |BufNewFile| starting to edit a file that doesn't exist |BufReadPre| starting to edit a new buffer, before reading the file |BufRead| starting to edit a new buffer, after reading the file |BufReadPost| starting to edit a new buffer, after reading the file |BufReadCmd| before starting to edit a new buffer |Cmd-event| |FileReadPre| before reading a file with a ":read" command |FileReadPost| after reading a file with a ":read" command |FileReadCmd| before reading a file with a ":read" command |Cmd-event| |FilterReadPre| before reading a file from a filter command |FilterReadPost| after reading a file from a filter command |StdinReadPre| before reading from stdin into the buffer |StdinReadPost| After reading from the stdin into the buffer Writing |BufWrite| starting to write the whole buffer to a file |BufWritePre| starting to write the whole buffer to a file |BufWritePost| after writing the whole buffer to a file |BufWriteCmd| before writing the whole buffer to a file |Cmd-event| |FileWritePre| starting to write part of a buffer to a file |FileWritePost| after writing part of a buffer to a file |FileWriteCmd| before writing part of a buffer to a file |Cmd-event| |FileAppendPre| starting to append to a file |FileAppendPost| after appending to a file |FileAppendCmd| before appending to a file |Cmd-event| |FilterWritePre| starting to write a file for a filter command or diff |FilterWritePost| after writing a file for a filter command or diff Buffers |BufAdd| just after adding a buffer to the buffer list |BufCreate| just after adding a buffer to the buffer list |BufDelete| before deleting a buffer from the buffer list |BufWipeout| before completely deleting a buffer |BufFilePre| before changing the name of the current buffer |BufFilePost| after changing the name of the current buffer |BufEnter| after entering a buffer |BufLeave| before leaving to another buffer |BufWinEnter| after a buffer is displayed in a window |BufWinLeave| before a buffer is removed from a window |BufUnload| before unloading a buffer |BufHidden| just after a buffer has become hidden |BufNew| just after creating a new buffer |SwapExists| detected an existing swap file Options |FileType| when the 'filetype' option has been set |Syntax| when the 'syntax' option has been set |EncodingChanged| after the 'encoding' option has been changed |TermChanged| after the value of 'term' has changed Startup and exit |VimEnter| after doing all the startup stuff |GUIEnter| after starting the GUI successfully |TermResponse| after the terminal response to |t_RV| is received |VimLeavePre| before exiting Vim, before writing the viminfo file |VimLeave| before exiting Vim, after writing the viminfo file Various |FileChangedShell| Vim notices that a file changed since editing started |FileChangedShellPost| After handling a file changed since editing started |FileChangedRO| before making the first change to a read-only file |ShellCmdPost| after executing a shell command |ShellFilterPost| after filtering with a shell command |FuncUndefined| a user function is used but it isn't defined |SpellFileMissing| a spell file is used but it can't be found |SourcePre| before sourcing a Vim script |SourceCmd| before sourcing a Vim script |Cmd-event| |VimResized| after the Vim window size changed |FocusGained| Vim got input focus |FocusLost| Vim lost input focus |CursorHold| the user doesn't press a key for a while |CursorHoldI| the user doesn't press a key for a while in Insert mode |CursorMoved| the cursor was moved in Normal mode |CursorMovedI| the cursor was moved in Insert mode |WinEnter| after entering another window |WinLeave| before leaving a window |TabEnter| after entering another tab page |TabLeave| before leaving a tab page |CmdwinEnter| after entering the command-line window |CmdwinLeave| before leaving the command-line window |InsertEnter| starting Insert mode |InsertChange| when typing <Insert> while in Insert or Replace mode |InsertLeave| when leaving Insert mode |ColorScheme| after loading a color scheme |RemoteReply| a reply from a server Vim was received |QuickFixCmdPre| before a quickfix command is run |QuickFixCmdPost| after a quickfix command is run |SessionLoadPost| after loading a session file |MenuPopup| just before showing the popup menu |User| to be used in combination with ":doautocmd" The alphabetical list of autocommand events: *autocmd-events-abc* *BufCreate* *BufAdd* BufAdd or BufCreate Just after creating a new buffer which is added to the buffer list, or adding a buffer to the buffer list. Also used just after a buffer in the buffer list has been renamed. The BufCreate event is for historic reasons. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being created "<afile>". *BufDelete* BufDelete Before deleting a buffer from the buffer list. The BufUnload may be called first (if the buffer was loaded). Also used just before a buffer in the buffer list is renamed. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being deleted "<afile>" and "<abuf>". Don't change to another buffer, it will cause problems. *BufEnter* BufEnter After entering a buffer. Useful for setting options for a file type. Also executed when starting to edit a buffer, after the BufReadPost autocommands. *BufFilePost* BufFilePost After changing the name of the current buffer with the ":file" or ":saveas" command. *BufFilePre* BufFilePre Before changing the name of the current buffer with the ":file" or ":saveas" command. *BufHidden* BufHidden Just after a buffer has become hidden. That is, when there are no longer windows that show the buffer, but the buffer is not unloaded or deleted. Not used for ":qa" or ":q" when exiting Vim. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being unloaded "<afile>". *BufLeave* BufLeave Before leaving to another buffer. Also when leaving or closing the current window and the new current window is not for the same buffer. Not used for ":qa" or ":q" when exiting Vim. *BufNew* BufNew Just after creating a new buffer. Also used just after a buffer has been renamed. When the buffer is added to the buffer list BufAdd will be triggered too. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being created "<afile>". *BufNewFile* BufNewFile When starting to edit a file that doesn't exist. Can be used to read in a skeleton file. *BufRead* *BufReadPost* BufRead or BufReadPost When starting to edit a new buffer, after reading the file into the buffer, before executing the modelines. See |BufWinEnter| for when you need to do something after processing the modelines. This does NOT work for ":r file". Not used when the file doesn't exist. Also used after successfully recovering a file. *BufReadCmd* BufReadCmd Before starting to edit a new buffer. Should read the file into the buffer. |Cmd-event| *BufReadPre* *E200* *E201* BufReadPre When starting to edit a new buffer, before reading the file into the buffer. Not used if the file doesn't exist. *BufUnload* BufUnload Before unloading a buffer. This is when the text in the buffer is going to be freed. This may be after a BufWritePost and before a BufDelete. Also used for all buffers that are loaded when Vim is going to exit. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being unloaded "<afile>". Don't change to another buffer, it will cause problems. *BufWinEnter* BufWinEnter After a buffer is displayed in a window. This can be when the buffer is loaded (after processing the modelines) or when a hidden buffer is displayed in a window (and is no longer hidden). Does not happen for |:split| without arguments, since you keep editing the same buffer, or ":split" with a file that's already open in a window, because it re-uses an existing buffer. But it does happen for a ":split" with the name of the current buffer, since it reloads that buffer. *BufWinLeave* BufWinLeave Before a buffer is removed from a window. Not when it's still visible in another window. Also triggered when exiting. It's triggered before BufUnload or BufHidden. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being unloaded "<afile>". *BufWipeout* BufWipeout Before completely deleting a buffer. The BufUnload and BufDelete events may be called first (if the buffer was loaded and was in the buffer list). Also used just before a buffer is renamed (also when it's not in the buffer list). NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer being deleted "<afile>". Don't change to another buffer, it will cause problems. *BufWrite* *BufWritePre* BufWrite or BufWritePre Before writing the whole buffer to a file. *BufWriteCmd* BufWriteCmd Before writing the whole buffer to a file. Should do the writing of the file and reset 'modified' if successful, unless '+' is in 'cpo' and writing to another file |cpo-+|. The buffer contents should not be changed. |Cmd-event| *BufWritePost* BufWritePost After writing the whole buffer to a file (should undo the commands for BufWritePre). *CmdwinEnter* CmdwinEnter After entering the command-line window. Useful for setting options specifically for this special type of window. This is triggered _instead_ of BufEnter and WinEnter. <afile> is set to a single character, indicating the type of command-line. |cmdwin-char| *CmdwinLeave* CmdwinLeave Before leaving the command-line window. Useful to clean up any global setting done with CmdwinEnter. This is triggered _instead_ of BufLeave and WinLeave. <afile> is set to a single character, indicating the type of command-line. |cmdwin-char| *ColorScheme* ColorScheme After loading a color scheme. |:colorscheme| *CursorHold* CursorHold When the user doesn't press a key for the time specified with 'updatetime'. Not re-triggered until the user has pressed a key (i.e. doesn't fire every 'updatetime' ms if you leave Vim to make some coffee. :) See |CursorHold-example| for previewing tags. This event is only triggered in Normal mode. It is not triggered when waiting for a command argument to be typed, or a movement after an operator. While recording the CursorHold event is not triggered. |q| Note: Interactive commands cannot be used for this event. There is no hit-enter prompt, the screen is updated directly (when needed). Note: In the future there will probably be another option to set the time. Hint: to force an update of the status lines use: :let &ro = &ro {only on Amiga, Unix, Win32, MSDOS and all GUI versions} *CursorHoldI* CursorHoldI Just like CursorHold, but in Insert mode. *CursorMoved* CursorMoved After the cursor was moved in Normal mode. Also when the text of the cursor line has been changed, e.g., with "x", "rx" or "p". Not triggered when there is typeahead or when an operator is pending. For an example see |match-parens|. Careful: Don't do anything that the user does not expect or that is slow. *CursorMovedI* CursorMovedI After the cursor was moved in Insert mode. Otherwise the same as CursorMoved. *EncodingChanged* EncodingChanged Fires off after the 'encoding' option has been changed. Useful to set up fonts, for example. *FileAppendCmd* FileAppendCmd Before appending to a file. Should do the appending to the file. Use the '[ and '] marks for the range of lines.|Cmd-eventYXXY *FileAppendPost* FileAppendPost After appending to a file. *FileAppendPre* FileAppendPre Before appending to a file. Use the '[ and '] marks for the range of lines. *FileChangedRO* FileChangedRO Before making the first change to a read-only file. Can be used to check-out the file from a source control system. Not triggered when the change was caused by an autocommand. This event is triggered when making the first change in a buffer or the first change after 'readonly' was set, just before the change is applied to the text. WARNING: If the autocommand moves the cursor the effect of the change is undefined. *E788* It is not allowed to change to another buffer here. You can reload the buffer but not edit another one. *FileChangedShell* FileChangedShell When Vim notices that the modification time of a file has changed since editing started. Also when the file attributes of the file change. |timestamp| Mostly triggered after executing a shell command, but also with a |:checktime| command or when Gvim regains input focus. This autocommand is triggered for each changed file. It is not used when 'autoread' is set and the buffer was not changed. If a FileChangedShell autocommand is present the warning message and prompt is not given. The |v:fcs_reason| variable is set to indicate what happened and |v:fcs_choice| can be used to tell Vim what to do next. NOTE: When this autocommand is executed, the current buffer "%" may be different from the buffer that was changed "<afile>". NOTE: The commands must not change the current buffer, jump to another buffer or delete a buffer. *E246* NOTE: This event never nests, to avoid an endless loop. This means that while executing commands for the FileChangedShell event no other FileChangedShell event will be triggered. *FileChangedShellPost* FileChangedShellPost After handling a file that was changed outside of Vim. Can be used to update the statusline. *FileEncoding* FileEncoding Obsolete. It still works and is equivalent to |EncodingChanged|. *FileReadCmd* FileReadCmd Before reading a file with a ":read" command. Should do the reading of the file. |Cmd-event| *FileReadPost* FileReadPost After reading a file with a ":read" command. Note that Vim sets the '[ and '] marks to the first and last line of the read. This can be used to operate on the lines just read. *FileReadPre* FileReadPre Before reading a file with a ":read" command. *FileType* FileType When the 'filetype' option has been set. The pattern is matched against the filetype. <afile> can be used for the name of the file where this option was set, and <amatch> for the new value of 'filetype'. See |filetypes|. *FileWriteCmd* FileWriteCmd Before writing to a file, when not writing the whole buffer. Should do the writing to the file. Should not change the buffer. Use the '[ and '] marks for the range of lines. |Cmd-event| *FileWritePost* FileWritePost After writing to a file, when not writing the whole buffer. *FileWritePre* FileWritePre Before writing to a file, when not writing the whole buffer. Use the '[ and '] marks for the range of lines. *FilterReadPost* FilterReadPost After reading a file from a filter command. Vim checks the pattern against the name of the current buffer as with FilterReadPre. Not triggered when 'shelltemp' is off. *FilterReadPre* *E135* FilterReadPre Before reading a file from a filter command. Vim checks the pattern against the name of the current buffer, not the name of the temporary file that is the output of the filter command. Not triggered when 'shelltemp' is off. *FilterWritePost* FilterWritePost After writing a file for a filter command or making a diff. Vim checks the pattern against the name of the current buffer as with FilterWritePre. Not triggered when 'shelltemp' is off. *FilterWritePre* FilterWritePre Before writing a file for a filter command or making a diff. Vim checks the pattern against the name of the current buffer, not the name of the temporary file that is the output of the filter command. Not triggered when 'shelltemp' is off. *FocusGained* FocusGained When Vim got input focus. Only for the GUI version and a few console versions where this can be detected. *FocusLost* FocusLost When Vim lost input focus. Only for the GUI version and a few console versions where this can be detected. May also happen when a dialog pops up. *FuncUndefined* FuncUndefined When a user function is used but it isn't defined. Useful for defining a function only when it's used. The pattern is matched against the function name. Both <amatch> and <afile> are set to the name of the function. See |autoload-functions|. *GUIEnter* GUIEnter After starting the GUI successfully, and after opening the window. It is triggered before VimEnter when using gvim. Can be used to position the window from a .gvimrc file: :autocmd GUIEnter * winpos 100 50 *GUIFailed* GUIFailed After starting the GUI failed. Vim may continue to run in the terminal, if possible (only on Unix and alikes, when connecting the X server fails). You may want to quit Vim: :autocmd GUIFailed * qall *InsertChange* InsertChange When typing <Insert> while in Insert or Replace mode. The |v:insertmode| variable indicates the new mode. Be careful not to move the cursor or do anything else that the user does not expect. *InsertEnter* InsertEnter Just before starting Insert mode. Also for Replace mode and Virtual Replace mode. The |v:insertmode| variable indicates the mode. Be careful not to move the cursor or do anything else that the user does not expect. *InsertLeave* InsertLeave When leaving Insert mode. Also when using CTRL-O |i_CTRL-O|. But not for |i_CTRL-C|. *MenuPopup* MenuPopup Just before showing the popup menu (under the right mouse button). Useful for adjusting the menu for what is under the cursor or mouse pointer. The pattern is matched against a single character representing the mode: n Normal v Visual o Operator-pending i Insert c Command line *QuickFixCmdPre* QuickFixCmdPre Before a quickfix command is run (|:make|, |:lmake|, |:grep|, |:lgrep|, |:grepadd|, |:lgrepadd|, |:vimgrep|, |:lvimgrep|, |:vimgrepadd|, |:lvimgrepadd|). The pattern is matched against the command being run. When |:grep| is used but 'grepprg' is set to "internal" it still matches "grep". This command cannot be used to set the 'makeprg' and 'grepprg' variables. If this command causes an error, the quickfix command is not executed. *QuickFixCmdPost* QuickFixCmdPost Like QuickFixCmdPre, but after a quickfix command is run, before jumping to the first location. See |QuickFixCmdPost-example|. *RemoteReply* RemoteReply When a reply from a Vim that functions as server was received |server2client()|. The pattern is matched against the {serverid}. <amatch> is equal to the {serverid} from which the reply was sent, and <afile> is the actual reply string. Note that even if an autocommand is defined, the reply should be read with |remote_read()| to consume it. *SessionLoadPost* SessionLoadPost After loading the session file created using the |:mksession| command. *ShellCmdPost* ShellCmdPost After executing a shell command with |:!cmd|, |:shell|, |:make| and |:grep|. Can be used to check for any changed files. *ShellFilterPost* ShellFilterPost After executing a shell command with ":{range}!cmd", ":w !cmd" or ":r !cmd". Can be used to check for any changed files. *SourcePre* SourcePre Before sourcing a Vim script. |:source| <afile> is the name of the file being sourced. *SourceCmd* SourceCmd When sourcing a Vim script. |:source| <afile> is the name of the file being sourced. The autocommand must source this file. |Cmd-event| *SpellFileMissing* SpellFileMissing When trying to load a spell checking file and it can't be found. The pattern is matched against the language. <amatch> is the language, 'encoding' also matters. See |spell-SpellFileMissing|. *StdinReadPost* StdinReadPost After reading from the stdin into the buffer, before executing the modelines. Only used when the "-" argument was used when Vim was started |--|. *StdinReadPre* StdinReadPre Before reading from stdin into the buffer. Only used when the "-" argument was used when Vim was started |--|. *SwapExists* SwapExists Detected an existing swap file when starting to edit a file. Only when it is possible to select a way to handle the situation, when Vim would ask the user what to do. The |v:swapname| variable holds the name of the swap file found, <afile> the file being edited. |v:swapcommand| may contain a command to be executed in the opened file. The commands should set the |v:swapchoice| variable to a string with one character to tell Vim what should be done next: 'o' open read-only 'e' edit the file anyway 'r' recover 'd' delete the swap file 'q' quit, don't edit the file 'a' abort, like hitting CTRL-C When set to an empty string the user will be asked, as if there was no SwapExists autocmd. *E812* It is not allowed to change to another buffer, change a buffer name or change directory here. *Syntax* Syntax When the 'syntax' option has been set. The pattern is matched against the syntax name. <afile> can be used for the name of the file where this option was set, and <amatch> for the new value of 'syntax'. See |:syn-on|. *TabEnter* TabEnter Just after entering a tab page. |tab-page| After triggering the WinEnter and before triggering the BufEnter event. *TabLeave* TabLeave Just before leaving a tab page. |tab-page| A WinLeave event will have been triggered first. *TermChanged* TermChanged After the value of 'term' has changed. Useful for re-loading the syntax file to update the colors, fonts and other terminal-dependent settings. Executed for all loaded buffers. *TermResponse* TermResponse After the response to |t_RV| is received from the terminal. The value of |v:termresponse| can be used to do things depending on the terminal version. *User* User Never executed automatically. To be used for autocommands that are only executed with ":doautocmd". *UserGettingBored* UserGettingBored When the user hits CTRL-C. Just kidding! :-) *VimEnter* VimEnter After doing all the startup stuff, including loading .vimrc files, executing the "-c cmd" arguments, creating all windows and loading the buffers in them. *VimLeave* VimLeave Before exiting Vim, just after writing the .viminfo file. Executed only once, like VimLeavePre. To detect an abnormal exit use |v:dying|. *VimLeavePre* VimLeavePre Before exiting Vim, just before writing the .viminfo file. This is executed only once, if there is a match with the name of what happens to be the current buffer when exiting. Mostly useful with a "*" pattern. :autocmd VimLeavePre * call CleanupStuff() To detect an abnormal exit use |v:dying|. *VimResized* VimResized After the Vim window was resized, thus 'lines' and/or 'columns' changed. Not when starting up though. *WinEnter* WinEnter After entering another window. Not done for the first window, when Vim has just started. Useful for setting the window height. If the window is for another buffer, Vim executes the BufEnter autocommands after the WinEnter autocommands. Note: When using ":split fname" the WinEnter event is triggered after the split but before the file "fname" is loaded. *WinLeave* WinLeave Before leaving a window. If the window to be entered next is for a different buffer, Vim executes the BufLeave autocommands before the WinLeave autocommands (but not for ":new"). Not used for ":qa" or ":q" when exiting Vim.
6. Patterns *autocmd-patterns* *{pat}* The file pattern {pat} is tested for a match against the file name in one of two ways: 1. When there is no '/' in the pattern, Vim checks for a match against only the tail part of the file name (without its leading directory path). 2. When there is a '/' in the pattern, Vim checks for a match against the both short file name (as you typed it) and the full file name (after expanding it to a full path and resolving symbolic links). The special pattern <buffer> or <buffer=N> is used for buffer-local autocommands |autocmd-buflocal|. This pattern is not matched against the name of a buffer. Examples: :autocmd BufRead *.txt set et Set the 'et' option for all text files. :autocmd BufRead /vim/src/*.c set cindent Set the 'cindent' option for C files in the /vim/src directory. :autocmd BufRead /tmp/*.c set ts=5 If you have a link from "/tmp/test.c" to "/home/nobody/vim/src/test.c", and you start editing "/tmp/test.c", this autocommand will match. Note: To match part of a path, but not from the root directory, use a '*' as the first character. Example: :autocmd BufRead */doc/*.txt set tw=78 This autocommand will for example be executed for "/tmp/doc/xx.txt" and "/usr/home/piet/doc/yy.txt". The number of directories does not matter here. The file name that the pattern is matched against is after expanding wildcards. Thus if you issue this command: :e $ROOTDIR/main.$EXT The argument is first expanded to: /usr/root/main.py Before it's matched with the pattern of the autocommand. Careful with this when using events like FileReadCmd, the value of <amatch> may not be what you expect. Environment variables can be used in a pattern: :autocmd BufRead $VIMRUNTIME/doc/*.txt set expandtab And ~ can be used for the home directory (if $HOME is defined): :autocmd BufWritePost ~/.vimrc so ~/.vimrc :autocmd BufRead ~archive/* set readonly The environment variable is expanded when the autocommand is defined, not when the autocommand is executed. This is different from the command! *file-pattern* The pattern is interpreted like mostly used in file names: * matches any sequence of characters ? matches any single character \? matches a '?' . matches a '.' ~ matches a '~' , separates patterns \, matches a ',' { } like \( \) in a |pattern| , inside { }: like \| in a |pattern|||| \ special meaning like in a |pattern| [ch] matches 'c' or 'h' [^ch] match any character but 'c' and 'h' Note that for all systems the '/' character is used for path separator (even MS-DOS and OS/2). This was done because the backslash is difficult to use in a pattern and to make the autocommands portable across different systems. *autocmd-changes* Matching with the pattern is done when an event is triggered. Changing the buffer name in one of the autocommands, or even deleting the buffer, does not change which autocommands will be executed. Example: au BufEnter *.foo bdel au BufEnter *.foo set modified This will delete the current buffer and then set 'modified' in what has become the current buffer instead. Vim doesn't take into account that "*.foo" doesn't match with that buffer name. It matches "*.foo" with the name of the buffer at the moment the event was triggered. However, buffer-local autocommands will not be executed for a buffer that has been wiped out with |:bwipe|. After deleting the buffer with |:bdel| the buffer actually still exists (it becomes unlisted), thus the autocommands are still executed.
7. Buffer-local autocommands *autocmd-buflocal* *autocmd-buffer-local* *<buffer=N>* *<buffer=abuf>* *E680* Buffer-local autocommands are attached to a specific buffer. They are useful if the buffer does not have a name and when the name does not match a specific pattern. But it also means they must be explicitly added to each buffer. Instead of a pattern buffer-local autocommands use one of these forms: <buffer> current buffer <buffer=99> buffer number 99 <buffer=abuf> using <abuf> (only when executing autocommands) |<abuf>| Examples: :au CursorHold <buffer> echo 'hold' :au CursorHold <buffer=33> echo 'hold' :au CursorHold <buffer=abuf> echo 'hold' All the commands for autocommands also work with buffer-local autocommands, simply use the special string instead of the pattern. Examples: :au! * <buffer> " remove buffer-local autocommands for " current buffer :au! * <buffer=33> " remove buffer-local autocommands for " buffer #33 :bufdo :au! CursorHold <buffer> " remove autocmd for given event for all " buffers :au * <buffer> " list buffer-local autocommands for " current buffer Note that when an autocommand is defined for the current buffer, it is stored with the buffer number. Thus it uses the form "<buffer=12>", where 12 is the number of the current buffer. You will see this when listing autocommands, for example. To test for presence of buffer-local autocommands use the |exists()| function as follows: :if exists("#CursorHold#<buffer=12>") | ... | endif :if exists("#CursorHold#<buffer>") | ... | endif " for current buffer When a buffer is wiped out its buffer-local autocommands are also gone, of course. Note that when deleting a buffer, e.g., with ":bdel", it is only unlisted, the autocommands are still present. In order to see the removal of buffer-local autocommands: :set verbose=6 It is not possible to define buffer-local autocommands for a non-existent buffer.
8. Groups *autocmd-groups* Autocommands can be put together in a group. This is useful for removing or executing a group of autocommands. For example, all the autocommands for syntax highlighting are put in the "highlight" group, to be able to execute ":doautoall highlight BufRead" when the GUI starts. When no specific group is selected, Vim uses the default group. The default group does not have a name. You cannot execute the autocommands from the default group separately; you can execute them only by executing autocommands for all groups. Normally, when executing autocommands automatically, Vim uses the autocommands for all groups. The group only matters when executing autocommands with ":doautocmd" or ":doautoall", or when defining or deleting autocommands. The group name can contain any characters except white space. The group name "end" is reserved (also in uppercase). The group name is case sensitive. Note that this is different from the event name! *:aug* *:augroup* :aug[roup] {name} Define the autocmd group name for the following ":autocmd" commands. The name "end" or "END" selects the default group. *:augroup-delete* *E367* :aug[roup]! {name} Delete the autocmd group {name}. Don't use this if there is still an autocommand using this group! This is not checked. To enter autocommands for a specific group, use this method: 1. Select the group with ":augroup {name}". 2. Delete any old autocommands with ":au!". 3. Define the autocommands. 4. Go back to the default group with "augroup END". Example: :augroup uncompress : au! : au BufEnter *.gz %!gunzip :augroup END This prevents having the autocommands defined twice (e.g., after sourcing the .vimrc file again).
9. Executing autocommands *autocmd-execute* Vim can also execute Autocommands non-automatically. This is useful if you have changed autocommands, or when Vim has executed the wrong autocommands (e.g., the file pattern match was wrong). Note that the 'eventignore' option applies here too. Events listed in this option will not cause any commands to be executed. *:do* *:doau* *:doautocmd* *E217* :do[autocmd] [group] {event} [fname] Apply the autocommands matching [fname] (default: current file name) for {event} to the current buffer. You can use this when the current file name does not match the right pattern, after changing settings, or to execute autocommands for a certain event. It's possible to use this inside an autocommand too, so you can base the autocommands for one extension on another extension. Example: :au Bufenter *.cpp so ~/.vimrc_cpp :au Bufenter *.cpp doau BufEnter x.c Be careful to avoid endless loops. See |autocmd-nested|. When the [group] argument is not given, Vim executes the autocommands for all groups. When the [group] argument is included, Vim executes only the matching autocommands for that group. Note: if you use an undefined group name, Vim gives you an error message. After applying the autocommands the modelines are processed, so that their settings overrule the settings from autocommands, like what happens when editing a file. *:doautoa* *:doautoall* :doautoa[ll] [group] {event} [fname] Like ":doautocmd", but apply the autocommands to each loaded buffer. Note that {fname} is used to select the autocommands, not the buffers to which they are applied. Careful: Don't use this for autocommands that delete a buffer, change to another buffer or change the contents of a buffer; the result is unpredictable. This command is intended for autocommands that set options, change highlighting, and things like that.
10. Using autocommands *autocmd-use* For WRITING FILES there are four possible sets of events. Vim uses only one of these sets for a write command: BufWriteCmd BufWritePre BufWritePost writing the whole buffer FilterWritePre FilterWritePost writing to filter temp file FileAppendCmd FileAppendPre FileAppendPost appending to a file FileWriteCmd FileWritePre FileWritePost any other file write When there is a matching "*Cmd" autocommand, it is assumed it will do the writing. No further writing is done and the other events are not triggered. |Cmd-event| Note that the *WritePost commands should undo any changes to the buffer that were caused by the *WritePre commands; otherwise, writing the file will have the side effect of changing the buffer. Before executing the autocommands, the buffer from which the lines are to be written temporarily becomes the current buffer. Unless the autocommands change the current buffer or delete the previously current buffer, the previously current buffer is made the current buffer again. The *WritePre and *AppendPre autocommands must not delete the buffer from which the lines are to be written. The '[ and '] marks have a special position: - Before the *ReadPre event the '[ mark is set to the line just above where the new lines will be inserted. - Before the *ReadPost event the '[ mark is set to the first line that was just read, the '] mark to the last line. - Before executing the *WriteCmd, *WritePre and *AppendPre autocommands the '[ mark is set to the first line that will be written, the '] mark to the last line. Careful: '[ and '] change when using commands that change the buffer. In commands which expect a file name, you can use "<afile>" for the file name that is being read |:<afile>| (you can also use "%" for the current file name). "<abuf>" can be used for the buffer number of the currently effective buffer. This also works for buffers that doesn't have a name. But it doesn't work for files without a buffer (e.g., with ":r file"). *gzip-example* Examples for reading and writing compressed files: :augroup gzip : autocmd! : autocmd BufReadPre,FileReadPre *.gz set bin : autocmd BufReadPost,FileReadPost *.gz '[,']!gunzip : autocmd BufReadPost,FileReadPost *.gz set nobin : autocmd BufReadPost,FileReadPost *.gz execute ":doautocmd BufReadPost " . expand("%:r") : autocmd BufWritePost,FileWritePost *.gz !mv <afile> <afile>:r : autocmd BufWritePost,FileWritePost *.gz !gzip <afile>:r : autocmd FileAppendPre *.gz !gunzip <afile> : autocmd FileAppendPre *.gz !mv <afile>:r <afile> : autocmd FileAppendPost *.gz !mv <afile> <afile>:r : autocmd FileAppendPost *.gz !gzip <afile>:r :augroup END The "gzip" group is used to be able to delete any existing autocommands with ":autocmd!", for when the file is sourced twice. ("<afile>:r" is the file name without the extension, see |:_%:|) The commands executed for the BufNewFile, BufRead/BufReadPost, BufWritePost, FileAppendPost and VimLeave events do not set or reset the changed flag of the buffer. When you decompress the buffer with the BufReadPost autocommands, you can still exit with ":q". When you use ":undo" in BufWritePost to undo the changes made by BufWritePre commands, you can still do ":q" (this also makes "ZZ" work). If you do want the buffer to be marked as modified, set the 'modified' option. To execute Normal mode commands from an autocommand, use the ":normal" command. Use with care! If the Normal mode command is not finished, the user needs to type characters (e.g., after ":normal m" you need to type a mark name). If you want the buffer to be unmodified after changing it, reset the 'modified' option. This makes it possible to exit the buffer with ":q" instead of ":q!". *autocmd-nested* *E218* By default, autocommands do not nest. If you use ":e" or ":w" in an autocommand, Vim does not execute the BufRead and BufWrite autocommands for those commands. If you do want this, use the "nested" flag for those commands in which you want nesting. For example: :autocmd FileChangedShell *.c nested e! The nesting is limited to 10 levels to get out of recursive loops. It's possible to use the ":au" command in an autocommand. This can be a self-modifying command! This can be useful for an autocommand that should execute only once. If you want to skip autocommands for one command, use the |:noautocmd| command modifier or the 'eventignore' option. Note: When reading a file (with ":read file" or with a filter command) and the last line in the file does not have an <EOL>, Vim remembers this. At the next write (with ":write file" or with a filter command), if the same line is written again as the last line in a file AND 'binary' is set, Vim does not supply an <EOL>. This makes a filter command on the just read lines write the same file as was read, and makes a write command on just filtered lines write the same file as was read from the filter. For example, another way to write a compressed file: :autocmd FileWritePre *.gz set bin|'[,']!gzip :autocmd FileWritePost *.gz undo|set nobin *autocommand-pattern* You can specify multiple patterns, separated by commas. Here are some examples: :autocmd BufRead * set tw=79 nocin ic infercase fo=2croq :autocmd BufRead .letter set tw=72 fo=2tcrq :autocmd BufEnter .letter set dict=/usr/lib/dict/words :autocmd BufLeave .letter set dict= :autocmd BufRead,BufNewFile *.c,*.h set tw=0 cin noic :autocmd BufEnter *.c,*.h abbr FOR for (i = 0; i < 3; ++i)<CR>{<CR>}<Esc>O :autocmd BufLeave *.c,*.h unabbr FOR For makefiles (makefile, Makefile, imakefile, makefile.unix, etc.): :autocmd BufEnter ?akefile* set include=^s\=include :autocmd BufLeave ?akefile* set include& To always start editing C files at the first function: :autocmd BufRead *.c,*.h 1;/^{ Without the "1;" above, the search would start from wherever the file was entered, rather than from the start of the file. *skeleton* *template* To read a skeleton (template) file when opening a new file: :autocmd BufNewFile *.c 0r ~/vim/skeleton.c :autocmd BufNewFile *.h 0r ~/vim/skeleton.h :autocmd BufNewFile *.java 0r ~/vim/skeleton.java To insert the current date and time in a *.html file when writing it: :autocmd BufWritePre,FileWritePre *.html ks|call LastMod()|'s :fun LastMod() : if line("$") > 20 : let l = 20 : else : let l = line("$") : endif : exe "1," . l . "g/Last modified: /s/Last modified: .*/Last modified: " . : \ strftime("%Y %b %d") :endfun You need to have a line "Last modified: <date time>" in the first 20 lines of the file for this to work. Vim replaces <date time> (and anything in the same line after it) with the current date and time. Explanation: ks mark current position with mark 's' call LastMod() call the LastMod() function to do the work 's return the cursor to the old position The LastMod() function checks if the file is shorter than 20 lines, and then uses the ":g" command to find lines that contain "Last modified: ". For those lines the ":s" command is executed to replace the existing date with the current one. The ":execute" command is used to be able to use an expression for the ":g" and ":s" commands. The date is obtained with the strftime() function. You can change its argument to get another date string. When entering :autocmd on the command-line, completion of events and command names may be done (with <Tab>, CTRL-D, etc.) where appropriate. Vim executes all matching autocommands in the order that you specify them. It is recommended that your first autocommand be used for all files by using "*" as the file pattern. This means that you can define defaults you like here for any settings, and if there is another matching autocommand it will override these. But if there is no other matching autocommand, then at least your default settings are recovered (if entering this file from another for which autocommands did match). Note that "*" will also match files starting with ".", unlike Unix shells. *autocmd-searchpat* Autocommands do not change the current search patterns. Vim saves the current search patterns before executing autocommands then restores them after the autocommands finish. This means that autocommands do not affect the strings highlighted with the 'hlsearch' option. Within autocommands, you can still use search patterns normally, e.g., with the "n" command. If you want an autocommand to set the search pattern, such that it is used after the autocommand finishes, use the ":let @/ =" command. The search-highlighting cannot be switched off with ":nohlsearch" in an autocommand. Use the 'h' flag in the 'viminfo' option to disable search- highlighting when starting Vim. *Cmd-event* When using one of the "*Cmd" events, the matching autocommands are expected to do the file reading, writing or sourcing. This can be used when working with a special kind of file, for example on a remote system. CAREFUL: If you use these events in a wrong way, it may have the effect of making it impossible to read or write the matching files! Make sure you test your autocommands properly. Best is to use a pattern that will never match a normal file name, for example "ftp://*". When defining a BufReadCmd it will be difficult for Vim to recover a crashed editing session. When recovering from the original file, Vim reads only those parts of a file that are not found in the swap file. Since that is not possible with a BufReadCmd, use the |:preserve| command to make sure the original file isn't needed for recovery. You might want to do this only when you expect the file to be modified. For file read and write commands the |v:cmdarg| variable holds the "++enc=" and "++ff=" argument that are effective. These should be used for the command that reads/writes the file. The |v:cmdbang| variable is one when "!" was used, zero otherwise. See the $VIMRUNTIME/plugin/netrw.vim for examples.
11. Disabling autocommands *autocmd-disable* To disable autocommands for some time use the 'eventignore' option. Note that this may cause unexpected behavior, make sure you restore 'eventignore' afterwards, using a |:try| block with |:finally|. *:noautocmd* *:noa* To disable autocommands for just one command use the ":noautocmd" command modifier. This will set 'eventignore' to "all" for the duration of the following command. Example: :noautocmd w fname.gz This will write the file without triggering the autocommands defined by the gzip plugin. top - main help file


A UICollectionViewController with list layout configuration
UICollectionViewController
In Reveal you can see how the implementation detail of a collection view's view hierarchy


Uses of Package
net.sf.jasperreports.export
-
Packages that use net.sf.jasperreports.export Package Description net.sf.jasperreports.engine Provides access to the library's main functionality through facade classes for compiling, filling and exporting reports and also publishes the library's main interfaces and classes.net.sf.jasperreports.engine.export Provides utility classes for exporting reports to various popular formats such as PDF, HTML, RTF, CSV, Excel, DOCX, PPTX, ODT, ODS, XML, Text, etc.net.sf.jasperreports.engine.export.oasis Provides utility classes for exporting reports to Open Document Format, such as ODT and ODS.net.sf.jasperreports.engine.export.ooxml Provides utility classes for exporting reports to Office Open XML Format, such as DOCX, XLSX and PPTX.net.sf.jasperreports.export Provides exporter input, exporter output and exporter configurationsnet.sf.jasperreports.export.parameters Provides support for export parametersnet.sf.jasperreports.j2ee.servlets Provides support for built-in JasperReports servlets. -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.engine Class Description Exporter All document exporting in JasperReports is done through this interface.ExporterConfiguration Instances of this class are used for configuring the behavior of exporters.ExporterInput ExporterInputItem ExporterOutput ReportExportConfiguration Instances of this class are used for configuring the behavior of exporters. -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.engine.export Class Description CsvExporterConfiguration Interface containing settings used by the CSV exporters.CsvMetadataExporterConfiguration Interface containing settings used by the metadata based CSV exporter.CsvMetadataReportConfiguration Interface containing settings used by the metadata based CSV exporter.CsvReportConfiguration Interface containing settings used by the CSV exporters.Exporter All document exporting in JasperReports is done through this interface.ExporterConfiguration Instances of this class are used for configuring the behavior of exporters.ExporterInputItem Graphics2DExporterConfiguration Interface containing settings used by the Graphics2D exporter.Graphics2DReportConfiguration Interface containing settings used by the Graphics2D exporter.HtmlExporterConfiguration Interface containing settings used by the HTML exporters.HtmlReportConfiguration Interface containing settings used by the HTML exporters.JsonExporterConfiguration Interface containing settings used by the JSON exporter.JsonMetadataReportConfiguration Interface containing settings used by the JSON Metadata exporter.JsonReportConfiguration Interface containing settings used by the JSON exporter.PdfExporterConfiguration Interface containing settings used by the PDF exporter.PdfReportConfiguration Interface containing settings used by the PDF exporter.PrintServiceExporterConfiguration Interface containing settings used by the print service exporter.PrintServiceReportConfiguration Interface containing settings used by the print service exporter.ReportExportConfiguration Instances of this class are used for configuring the behavior of exporters.RtfExporterConfiguration Interface containing settings used by the RTF exporter.RtfReportConfiguration Interface containing settings used by the RTF exporter.SimpleGraphics2DReportConfiguration TextExporterConfiguration Interface containing settings used by the pure text exporter.TextReportConfiguration Interface containing settings used by the pure text exporter.XlsExporterConfiguration Interface containing settings used by the Excel exporters.XlsMetadataExporterConfiguration Interface containing settings used by the Excel metadata exporters.XlsMetadataReportConfiguration Interface containing settings used by the Excel metadata exporters.XlsReportConfiguration Interface containing settings used by the Excel exporters. -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.engine.export.oasis Class Description Exporter All document exporting in JasperReports is done through this interface.OdsExporterConfiguration Interface containing settings used by the ODS exporter.OdsReportConfiguration Interface containing settings used by the ODS exporter.OdtExporterConfiguration Interface containing settings used by the ODT exporter.OdtReportConfiguration Interface containing settings used by the ODT exporter. -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.engine.export.ooxml Class Description DocxExporterConfiguration Interface containing settings used by the DOCX exporter.DocxReportConfiguration Interface containing settings used by the DOCX exporter.Exporter All document exporting in JasperReports is done through this interface.ExporterInput PptxExporterConfiguration Interface containing settings used by the PPTX exporter.PptxReportConfiguration Interface containing settings used by the PPTX exporter.XlsReportConfiguration Interface containing settings used by the Excel exporters.XlsxExporterConfiguration Interface containing settings used by the XLSX exporter.XlsxReportConfiguration Interface containing settings used by the XLSX exporter. -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.export Class Description AbstractXlsExporterConfiguration AbstractXlsReportConfiguration CommonExportConfiguration CsvExporterConfiguration Interface containing settings used by the CSV exporters.CsvMetadataExporterConfiguration Interface containing settings used by the metadata based CSV exporter.CsvMetadataReportConfiguration Interface containing settings used by the metadata based CSV exporter.CsvReportConfiguration Interface containing settings used by the CSV exporters.DocxExporterConfiguration Interface containing settings used by the DOCX exporter.DocxReportConfiguration Interface containing settings used by the DOCX exporter.ExporterConfiguration Instances of this class are used for configuring the behavior of exporters.ExporterInput ExporterInputItem ExporterOutput Graphics2DExporterConfiguration Interface containing settings used by the Graphics2D exporter.Graphics2DExporterOutput Graphics2DReportConfiguration Interface containing settings used by the Graphics2D exporter.HtmlExporterConfiguration Interface containing settings used by the HTML exporters.HtmlExporterOutput HtmlReportConfiguration Interface containing settings used by the HTML exporters.JsonExporterConfiguration Interface containing settings used by the JSON exporter.JsonExporterOutput JsonMetadataReportConfiguration Interface containing settings used by the JSON Metadata exporter.JsonReportConfiguration Interface containing settings used by the JSON exporter.OdsExporterConfiguration Interface containing settings used by the ODS exporter.OdsReportConfiguration Interface containing settings used by the ODS exporter.OdtExporterConfiguration Interface containing settings used by the ODT exporter.OdtReportConfiguration Interface containing settings used by the ODT exporter.OutputStreamExporterOutput PdfExporterConfiguration Interface containing settings used by the PDF exporter.PdfReportConfiguration Interface containing settings used by the PDF exporter.PptxExporterConfiguration Interface containing settings used by the PPTX exporter.PptxReportConfiguration Interface containing settings used by the PPTX exporter.PrintServiceExporterConfiguration Interface containing settings used by the print service exporter.PrintServiceReportConfiguration Interface containing settings used by the print service exporter.ReportExportConfiguration Instances of this class are used for configuring the behavior of exporters.RtfExporterConfiguration Interface containing settings used by the RTF exporter.RtfReportConfiguration Interface containing settings used by the RTF exporter.SimpleCommonExportConfiguration SimpleCsvExporterConfiguration SimpleCsvReportConfiguration SimpleExporterConfiguration SimpleExporterInput SimpleGraphics2DExporterConfiguration SimpleGraphics2DReportConfiguration SimpleJsonReportConfiguration SimpleReportExportConfiguration SimpleWriterExporterOutput SimpleXlsExporterConfiguration SimpleXlsReportConfiguration TextExporterConfiguration Interface containing settings used by the pure text exporter.TextReportConfiguration Interface containing settings used by the pure text exporter.WriterExporterOutput XlsExporterConfiguration Interface containing settings used by the Excel exporters.XlsMetadataExporterConfiguration Interface containing settings used by the Excel metadata exporters.XlsMetadataReportConfiguration Interface containing settings used by the Excel metadata exporters.XlsReportConfiguration Interface containing settings used by the Excel exporters.XlsxExporterConfiguration Interface containing settings used by the XLSX exporter.XlsxReportConfiguration Interface containing settings used by the XLSX exporter.XmlExporterOutput -
Classes in net.sf.jasperreports.export used by net.sf.jasperreports.j2ee.servlets Class Description XlsExporterConfiguration Interface containing settings used by the Excel exporters.XlsReportConfiguration Interface containing settings used by the Excel exporters.


Machine Learning Summer Schools
This post is the first in a series of posts highlighting my attempt(s) to prepare myself to undertake research and for graduate school. It will include a year in review, learnings from working at CERN, and some technical posts about the kind of work I do.
Background
I’m a recent graduate of Computer Engineering working at the European Organisation for Nuclear Research (CERN). I got into a “US top-15” school for pursuing a Masters in Computer Science starting 2018 but chose to work as a Technical Student for a year instead. My goal in spending this year at CERN is to to gain more perspective and better understand my interests before diving straight into graduate school. As I worked on projects at the CMS Experiment, I really developed a strong liking for problems we face in high-energy physics. This series of posts will serve as a reflection of the people I have met and experiences I have shared over the past summer.
Applying to Summer Schools
The year started on a mixed note; I was excited about working at CERN, but also wondering what directions to take in order to prepare for graduate studies. My aim is to learn by doing and make the most of opportunities that come along. So I decided to figure out how to improve my knowledge of machine learning that was essentially self-taught—taking online courses and working on a project aiding the deployment Tensorflow models in production on the CMS Software Environment. I spent two weeks researching and applying to machine learning summer schools with my supervisor (and former supervisor) supporting me with reference letters wherever necessary.
The schools had some really cool speakers lined up which I thought would make for a great introduction to subjects like gaussian processes, natural language processing, causal inference, probabilistic methods, bayesian non-parametrics, and optimisation. The application process for the schools was fairly straightforward (often doing away with the reference letter) and encouraged students to present posters of their ongoing/completed projects. I ended up visiting two schools - Machine Learning in High-energy Physics Summer School (Oxford, UK) and Machine Learning Summer School (Madrid, Spain), apart from the Deep Learning Indaba (Stellenbosch, SA). My poster on our work (DeepJet) was presented at MLSS and the Indaba.
Note: I learnt later how presenting posters forms a critical part of the experience; it fosters discussions on a plethora of interconnected domains promoting quick and reliable knowledge transfer. As Jeff Dean said in a talk at the Deep Learning Indaba, “If I had a choice between reading one research paper in great depth or just skimming through ten papers, I would choose the latter because that gives me a broader idea of the problem space that I can then build upon.” Poster presentations accomplish pretty much the same goal and in addition foster collaboration and interest in relevant work by others you may not otherwise hear of.
Note: I also learnt later how lucky I was to have qualified for these events that are otherwise primarily targeted at an audience of Ph.D. students (except for the Deep Learning Indaba). This gave me an invaluable set of opportunities to connect with researchers and graduate students who were perfectly equipped to offer relevant advice to me (a lot of people can offer advice, but very little is often relevant).
Takeaways
Here’s a brief outline of each of these events along with what I learnt from each of them. In this post, I’ve focused on the Fourth Machine Learning in High-energy Physics Summer School.
MLHEP 2018:
The Machine Learning in High-energy Physics Summer School was organised by researchers from Yandex, Skoltech, and HSE at Oxford University in the UK. It is an event targeted at Ph.D. students and Professors in Physics seeking a foothold in machine learning to find applications to their respective problem statements. However, given that I was working on a project along similar lines, it served as a pretty good primer for me.
School Highlights
- The school began with two tracks—one slightly more advanced introduction and another for beginners to programming and data science.
- The lectures adhered to a (really useful) hands-on format where each talk was followed by a practical coding session teaching the practical uses of each model.
- The MLHEP Starter code was really well documented and overall quite thorough in its treatment of the subject. It should serve as a useful reference even for those who have not attended the event.
- There was a set of invited lectures (one per day) that was truly the highlight of each day. It made me wonder how they shortlisted speakers but whatever the process was, it clearly worked. From an introduction to Quantum Physics and Deep Learning to NVidia’s use of AI, to Advertisements on Oracle Data Cloud, and a detailed architectural breakdown of Google’s newly introduced Tensor Processing Unit - these lectures did not disappoint! The slides for each of these lectures has been kindly made available by the organisers.
The Codalab Challenge
- Amidst a packed (read 7:30 am - 7 pm) schedule that prevented me from visiting this popular vegan joint (sadly, it shut by 5 pm daily) we had yet another activity - a multi-phase Codalab Challenge on the Semantic Segmentation of LArTPC (Liquid Argon Time Projection Chamber) tracks! Phase 1 was now open to us and the rest would follow soon after.
- If you don’t understand much about the title, don’t be disappointed. I didn’t know much about it either. But I did know that we had free GPU Credit and some starter code, and when someone is so kind as to give you a free GPU and some machine learning models, it is only natural for your overfitting instincts to kick into overdrive. So overfitting aside, that’s exactly what I did.
- My first four attempts gave me exactly zero accuracy. I had no idea what I was doing wrong. After 14 hours of training (I woke up intermittently through the night to check up on the model) I realized I was not preprocessing the data correctly. Long story short, after a series of modifications to my code, I arrived upon a seriously crappy model and placed almost last on the leaderboard.
That was probably my breaking point.
- Now there were two undergrads at the school who didn’t really participate in the challenge (to my knowledge, and based on the leaderboard). The rest of the thirty-ish people were Physics Ph.D. Students. They understood exactly what a LArTPC was, in addition to having the same starter code and a reasonably good understanding of machine learning. Then there was me - the third undergrad with almost entirely self-taught knowledge of machine learning (I’ll write another post sometime about my opinion on the Indian Education System) and barely any understanding of the properties of the track data. Did I really expect myself to miraculously arrive at the optimal hyperparameters for this problem given that I was already drained after 12 hours of attending the school daily?
Then, I binged on a box of Pringles and reevaluated matters
- I was an undergrad among 2nd and 3rd year Ph.D. students (also some Professors) and I still managed to not only get in but also successfully understand most of the topics taught in the school.
- The very fact that I had a submission placed me in the top-30% of the school (many people decided against participating due to various issues including the time constraints and a limited understanding of the starter code).
- I did finally start understanding what LArTPC was about, after watching some introductory lectures. It wasn’t that complicated. And the history of the TPC was pretty cool (I like reading about the story behind things).
- I used the more complicated model - the Message-passing Neural Network instead of the Convolutional Neural Network simply because I wanted to ‘try something new.’ I managed my model fairly well by tweaking the learning rate, playing around with dropout to prevent overfitting, switching around optimizers, and adjusting the number of training epochs. The accuracy (to my recollection) was 80-something percent.
- Given that it was my first Kaggle-like challenge, this was a pretty great outcome. Giving up now would be a very silly idea. Either way, there was only a day more left. Instead, why not try to push harder for a day without any expectation and see what I get.
So I did.
- For the next three nights, I didn’t waste a lot of time on dinner, grabbing what I could from the nearest restaurant. I focused on reading up about the physics end of things and understood what exactly the challenge expected us to do. Then, I dug up some esoteric stuff on graph nets that led me to the conclusion that this was not going to be easy to pick up in a few hours.
- Earlier I had assumed the code would be easy to tweak since I was looking at it from an outsider’s perspective. On diving deeper, I realized that it was “messier” (arguably “over-modularized”) than I had thought. Custom functions were called that were defined in separate files and directories and overall it took me a bit of time to try and get it working. To be fair, this was really nice code, but in the time we had it was difficult to digest everything. So I figured out how to use the base architecture, hacked together a working model, and set it training while I read up on how message passing works.
- The next few days were basically low on sleep and high on Tensorboard and excitement. I find that I get really excited about the little things especially when I like what I’m doing. Although I didn’t expect to achieve much, what I was doing seemed really interesting so I spent time understanding (or rather hypothesizing) why some models gave me better accuracy and losses but didn’t really produce an output as good as others on the test set (overfitting, maybe?). A very naive way of looking at it is that there’s tradeoffs between the learning rate you use and the number of training epochs that you need to monitor as the losses go down and accuracy increases. The plot has slight bumps (in the wrong direciton) after a point that seem to tell you that you might be overfitting.
- I realized it wouldn’t be feasible to use all of the training data (I/O was waaaaay too slow) nor would it be easy to create batches currently given the time constraints and custom code (I just had a day left at that point and we still had lectures). These tradeoffs resulted in a lot of updates to my approach and the code, ultimately arriving at a set of models that seemed to do okay, placing somewhere near the upper half of the leaderboard.
- As the deadline loomed and I submitted my last model, I landed up 4th on the Phase 1 Leaderboard for the challenge with an accuracy of 90-something percent. This wasn’t the miracle win I expected (blame Hollywood for setting my expectations high) but it was undoubtedly a decent result for a first-timer given the aforementioned constraints.
- The next morning, the CodaLab Challenge winners were asked to present their models. Alex, who stood first on the leaderboard, came up to explain his entry.
- As I listened to Alex talk about how he visualized the data (used the CNN for visualizing the features? not clear on this) and tweaked the learning rate and a lot of jargon (but in the nice sense), I was very impressed with the work. Having nothing else to focus on as they announced the next winner, I started planning my journey back to Heathrow from Oxford - I didn’t want to miss my flight.
- A friend sitting next to me elbows me as he points to the leaderboard. “Isn’t that you?”
“What?”
“
smehta
, that’s your username right?” “Oh, right. Well, shit.” - Mother of God, I stood second. How? Well, you see the challenge had multiple phases, and they simply used the test data from the second phase (only slightly advanced from Phase 1) to evaluate the models. This resulted in some models not performing as well - surprisingly enough, mine did. They had told us this before, and I remember I had tweaked my training and preprocessing to accommodate for a better score overall, but I was so busy with training afterwards that I forgot they would consider the results of Phase 2.
“Can the user smehta
please come up and explain their approach?”
- I knew I wasn’t going to follow up Alex’s presentation with something really technical so I decided to do the easy thing - tell them exactly how I did it.
“I’ve never used a Message-passing Neural Network and I wanted to see how it works so that was my main motivation in participating in this challenge. My first few tries failed which was when I updated my preprocessing pipeline. The core principle I used to tweak my training parameters was basically what Andrew Ng said in Coursera Machine Learning, Week 4 - ‘If you are approaching a minima you need to adjust your number of training steps and learning rate such that you neither overshoot it nor converge too slow.’”
That, ladies and gentlemen, is what I call machine learning (sarcasm? maybe).
Stay tuned for more posts about experiences from the two other events that I had the privilege to attend this year - Machine Learning Summer School (Madrid), and the Deep Learning Indaba (South Africa).


This project is to demonstrate how well our machine learning model of linear regression is predicting.
This model is not intented to predict what the stock price will be in the future but wouldn't that be nice?
Lastly, since our machine learning was using closing prices, the extreme volatility in both upward and downward pricing made it more difficult in making an accurate prediction.


The Thesaurus Linguae Aegyptiae is a publication platform of Egyptian texts of pharaonic times made available by the project "Structure and Transformation in the Vocabulary of the Egyptian Language" (former Ancient Egyptian Dictionary Project) at the Berlin-Brandenburg Academy of Sciences and Humanities and the Saxon Academy of Sciences in Leipzig. The project collaborates with other projects who contribute electronic annotated texts, too. Within the Thesaurus, a digital corpus of Egyptian (including Demotic) texts have been released to the public for computer-assisted search. Lemmatization and morpho-syntactic annotation of the text material allow for specific research from lexical, philological, linguistic, and historico-cultural points of view.
INTF is a research unit at Münster University devoted to edit the Greek New Testament in several editiorial formats. Its most ambitious endeavor is the Editio Critica Maior aiming at completely representing the text tradition of the first millenium. As such the ECM is funded by the North Rhine-Westphalian Academy of Sciences, Humanities and the Arts. In order to foster and help organize the logistics of such a large scale project the New Testament Virtual Manuscript Room has been created within which all the editorial tasks can be performed in a seamless server based digital environment.
Digital Edition of the Coptic Old Testament is a project run by the Göttingen Academy of Sciences and Humanities. It aims at a complete reconstruction of the oldest Coptic translation of the Old Testament by collecting all the remaining fragments of largely dismembered codices and by digitally re-uniting and publishing them as a series of critical editions. A vital part of this research is the Coptic Old Testament Manuscript Room as a derivative of the NTVMR adapted to serve the creation of the Critical Digital Edition and Translation of the Sahidic Old Testament.
Coptic SCRIPTORIUM is an open-source, open-access platform for interdisciplinary and computational research in texts in the Coptic language, particularly the Sahidic dialect, created by Caroline T. Schroeder (University of the Pacific) and Amir Zeldes (Georgetown University).
The project Database and Dictionary of Greek Loanwords in Coptic aims to produce a comprehensive lexicographical compilation and description of Greek loanwords in Coptic and pre-Coptic Egyptian texts, as well as Arabic loanwords, across all dialects and text types, thus examining over 1,500 years of contact-induced langauge change of the Egyptian-Coptic language.
“PAThs” aims to provide an in-depth diachronic understanding and eﬀective representation of the geography of Coptic literary production, a corpus of texts written in the Coptic language, of almost exclusive religious contents, produced in Egypt between the 3rd and the 11th centuries. “PAThs” has the ambition to progressively become an exhaustive repertory of the Coptic literary manuscript tradition and therefore to turn into a pivotal resource for the study of Coptic literature.
The Collaborative Research Centre investigates constellations of education and religion in Greco-Roman religion, Judaism, Christianity and Islam from the 5th century BCE to the 13th century CE. Research is guided by the assumption that investigating the relationship of education and religion will provide deeper insights into cultural, social, and religious dynamics which were fundamental in these cultures and religions; moreover, this research agenda will also be instrumental in revealing the historical roots of contemporary debates about education and religion.
eTRAP, an early career research group, studies the linguistic and literary phenomenon that is text reuse with a particular focus on historical languages. More specifically, it looks at how ancient authors copied, alluded to, paraphrased and translated each other as they spread their knowledge in writing. It seeks to provide a basic understanding of the historical text reuse methodology, investigating text reuse on big data or, in other words, datasets that, owing to their size, cannot be manually processed.


basically:
git allows you to keep track of changes to the files in a folder
and synchronize those changes to the same folder somewhere else
this is called distributed version control
so lets give it a test drive!
demo 1
(improve an existing sample)
be patient with maintainers
handy (but dangerous) commands
when all else fails, you can always delete your fork and start over
this presentation
is on github!
/


Next: Inheritance of Methods, Previous: Keyword Arguments in Generic Functions and Methods, Up: Generic Functions and Methods
When a generic function is called with particular arguments, it must determine the code to execute. This code is called the effective method for those arguments. The effective method is a combination of the applicable methods in the generic function that calls some or all of the methods.
If a generic function is called and no methods are applicable, the generic function no-applicable-method is invoked, with the results from that call being used as the results of the call to the original generic function. Calling no-applicable-method takes precedence over checking for acceptable keyword arguments; see Keyword Arguments in Generic Functions and Methods.
When the effective method has been determined, it is invoked with the same arguments as were passed to the generic function. Whatever values it returns are returned as the values of the generic function.


In coding, if we want to store information about multiply objects, we can gather these objects into collections, like arrays and hashes.
For example, if you have a CD with 10 songs on it, you can create an array called cd, that will have 10 elements , each element will be accessible in this way:
cd[0]-first element
cd [1]-second element
, and so on. To set each element , yoiu can either access it individually, like cd[3]="We will rock you", or set all elements while initializing array:
cd=["song1', "song2", ... , song10].
Any time you can add new element, set it pointing to new value, or delete it. In case you will try to access the element that does not exist, like cd[14], Ruby will not through an error, but simply return nil, because nothing is there.
While it is possible to delete element anywhere inside the array, like so: cd.delete_at(5), doing so you have to keep in mind that all elements on the right from removed one( in our case, elemnts with indexes 6 to 9) will change their index number to (i-1).
Both Arrays and Hashes can hold objects of different types .
So, initializing new Array can be done so:
cd=[] cd=Array.new cd=['song1','song2','song3',...,'song4']
You can operate each element within Array one at a time, like :
array[i]=array[i]*3,
or you can use Array,s "map" method to iterate through each element within Array, changing it:
array.map!{|n| n*3}
Hashes are similar to Arrays, but you access each element not by its index, but by the corresponding key. Here is example of a Hash:
age={"Peter"=>25,"Anna"=>27,"Masha"=>18}
So, to access an elemnt(value), within a Hash, you will call for its key, like:
age["Peter"]
So, Hashes are used, when we don't want to access elements by their index numbers, but rather by some keywords (for example, in phonebooks, address books, grades at school and so on.)
Same as with Arrays, you will get nil if you will try to access key that hash does not contain(You will not be able to find phone number of a person that is not in a database).


A progressive Node.js framework for building efficient and scalable server-side applications.
Library - Nestjs implements a Library Management backend system.
# Install dependencies
$ yarn install
The .env_example
file in the current directory is likely omitting key credentials needed to get the backend services running. Please fill it by yourself.
# Development mode
$ yarn start
# Watch mode
$ yarn start:dev
# Production mode
$ yarn start:prod
# Render documentation
$ npx @compodoc/compodoc -p tsconfig.json -s
$ npx @compodoc/compodoc -s
## Test
```bash
# Unit tests
$ yarn test
# E2E tests
$ yarn test:e2e
# Test coverage
$ yarn test:cov


The sparklyr package provides a dplyr interface to Spark DataFrames as well as an R interface to Spark’s distributed machine learning pipelines. However, since Spark is a general-purpose cluster computing system there are many other R interfaces that could be built (e.g. interfaces to custom machine learning pipelines, interfaces to 3rd party Spark packages, etc.).
The facilities used internally by sparklyr for its dplyr and machine learning interfaces are available to extension packages. This guide describes how you can use these tools to create your own custom R interfaces to Spark.
Here’s an example of an extension function that calls the text file line counting function available via the SparkContext:
library(sparklyr)
count_lines <- function(sc, file) {
spark_context(sc) %>%
invoke("textFile", file, 1L) %>%
invoke("count")
}
The count_lines
function takes a spark_connection
(sc
) argument which enables it to obtain a reference to the SparkContext
object, and in turn call the textFile().count()
method.
You can use this function with an existing sparklyr connection as follows:
library(sparklyr)
sc <- spark_connect(master = "local")
count_lines(sc, "hdfs://path/data.csv")
Here are links to some additional examples of extension packages:
| Package | Description |
|---|---|
spark.sas7bdat |
Read in SAS data in parallel into Apache Spark. |
rsparkling |
Extension for using H2O machine learning algorithms against Spark Data Frames. |
sparkhello |
Simple example of including a custom JAR file within an extension package. |
rddlist |
Implements some methods of an R list as a Spark RDD (resilient distributed dataset). |
sparkwarc |
Load WARC files into Apache Spark with sparklyr. |
sparkavro |
Load Avro data into Spark with sparklyr. It is a wrapper of spark-avro |
crassy |
Connect to Cassandra with sparklyr using the Spark-Cassandra-Connector . |
sparklygraphs |
R interface for GraphFrames which aims to provide the functionality of GraphX. |
sparklyr.nested |
Extension for working with nested data. |
sparklyudf |
Simple example registering an Scala UDF within an extension package. |
Three classes are defined for representing the fundamental types of the R to Java bridge:
| Function | Description |
|---|---|
spark_connection |
Connection between R and the Spark shell process |
spark_jobj |
Instance of a remote Spark object |
spark_dataframe |
Instance of a remote Spark DataFrame object |
S3 methods are defined for each of these classes so they can be easily converted to or from objects that contain or wrap them. Note that for any given spark_jobj
it’s possible to discover the underlying spark_connection
.
There are several functions available for calling the methods of Java objects and static methods of Java classes:
| Function | Description |
|---|---|
invoke |
Call a method on an object |
invoke_new |
Create a new object by invoking a constructor |
invoke_static |
Call a static method on an object |
For example, to create a new instance of the java.math.BigInteger
class and then call the longValue()
method on it you would use code like this:
billionBigInteger <- invoke_new(sc, "java.math.BigInteger", "1000000000")
billion <- invoke(billionBigInteger, "longValue")
Note the sc
argument: that’s the spark_connection
object which is provided by the front-end package (e.g. sparklyr).
The previous example can be re-written to be more compact and clear using magrittr pipes:
billion <- sc %>%
invoke_new("java.math.BigInteger", "1000000000") %>%
invoke("longValue")
This syntax is similar to the method-chaining syntax often used with Scala code so is generally preferred.
Calling a static method of a class is also straightforward. For example, to call the Math::hypot()
static function you would use this code:
hypot <- sc %>%
invoke_static("java.lang.Math", "hypot", 10, 20)
Creating an extension typically consists of writing R wrapper functions for a set of Spark services. In this section we’ll describe the typical form of these functions as well as how to handle special types like Spark DataFrames.
Here’s the wrapper function for textFile().count()
which we defined earlier:
count_lines <- function(sc, file) {
spark_context(sc) %>%
invoke("textFile", file, 1L) %>%
invoke("count")
}
The count_lines
function takes a spark_connection
(sc
) argument which enables it to obtain a reference to the SparkContext
object, and in turn call the textFile().count()
method.
The following functions are useful for implementing wrapper functions of various kinds:
| Function | Description |
|---|---|
spark_connection |
Get the Spark connection associated with an object (S3) |
spark_jobj |
Get the Spark jobj associated with an object (S3) |
spark_dataframe |
Get the Spark DataFrame associated with an object (S3) |
spark_context |
Get the SparkContext for a spark_connection
|
hive_context |
Get the HiveContext for a spark_connection
|
spark_version |
Get the version of Spark (as a numeric_version ) for a spark_connection
|
The use of these functions is illustrated in this simple example:
analyze <- function(x, features) {
# normalize whatever we were passed (e.g. a dplyr tbl) into a DataFrame
df <- spark_dataframe(x)
# get the underlying connection so we can create new objects
sc <- spark_connection(df)
# create an object to do the analysis and call its `analyze` and `summary`
# methods (note that the df and features are passed to the analyze function)
summary <- sc %>%
invoke_new("com.example.tools.Analyzer") %>%
invoke("analyze", df, features) %>%
invoke("summary")
# return the results
summary
}
The first argument is an object that can be accessed using the Spark DataFrame API (this might be an actual reference to a DataFrame or could rather be a dplyr tbl
which has a DataFrame reference inside it).
After using the spark_dataframe
function to normalize the reference, we extract the underlying Spark connection associated with the data frame using the spark_connection
function. Finally, we create a new Analyzer
object, call it’s analyze
method with the DataFrame and list of features, and then call the summary
method on the results of the analysis.
Accepting a spark_jobj
or spark_dataframe
as the first argument of a function makes it very easy to incorporate into magrittr pipelines so this pattern is highly recommended when possible.
When creating R packages which implement interfaces to Spark you may need to include additional dependencies. Your dependencies might be a set of Spark Packages or might be a custom JAR file. In either case, you’ll need a way to specify that these dependencies should be included during the initialization of a Spark session. A Spark dependency is defined using the spark_dependency
function:
| Function | Description |
|---|---|
spark_dependency |
Define a Spark dependency consisting of JAR files and Spark packages |
Your extension package can specify it’s dependencies by implementing a function named spark_dependencies
within the package (this function should not be publicly exported). For example, let’s say you were creating an extension package named sparkds that needs to include a custom JAR as well as the Redshift and Apache Avro packages:
spark_dependencies <- function(spark_version, scala_version, ...) {
spark_dependency(
jars = c(
system.file(
sprintf("java/sparkds-%s-%s.jar", spark_version, scala_version),
package = "sparkds"
)
),
packages = c(
sprintf("com.databricks:spark-redshift_%s:0.6.0", scala_version),
sprintf("com.databricks:spark-avro_%s:2.0.1", scala_version)
)
)
}
.onLoad <- function(libname, pkgname) {
sparklyr::register_extension(pkgname)
}
The spark_version
argument is provided so that a package can support multiple Spark versions for it’s JARs. Note that the argument will include just the major and minor versions (e.g. 1.6
or 2.0
) and will not include the patch level (as JARs built for a given major/minor version are expected to work for all patch levels).
The scala_version
argument is provided so that a single package can support multiple Scala compiler versions for it’s JARs and packages (currently Scala 1.6 downloadable binaries are compiled with Scala 2.10 and Scala 2.0 downloadable binaries are compiled with Scala 2.11).
The ...
argument is unused but nevertheless should be included to ensure compatibility if new arguments are added to spark_dependencies
in the future.
The .onLoad
function registers your extension package so that it’s spark_dependencies
function will be automatically called when new connections to Spark are made via spark_connect
:
library(sparklyr)
library(sparkds)
sc <- spark_connect(master = "local")
The sparklyr package includes a utility function (compile_package_jars
) that will automatically compile a JAR file from your Scala source code for the required permutations of Spark and Scala compiler versions. To use the function just invoke it from the root directory of your R package as follows:
sparklyr::compile_package_jars()
Note that a prerequisite to calling compile_package_jars
is the installation of the Scala 2.10 and 2.11 compilers to one of the following paths:
See the sparkhello repository for a complete example of including a custom JAR within an extension package.
When including a JAR file within an R package distributed on CRAN, you should follow the guidelines provided in Writing R Extensions:
Java code is a special case: except for very small programs, .java files should be byte-compiled (to a .class file) and distributed as part of a .jar file: the conventional location for the .jar file(s) is
inst/java
. It is desirable (and required under an Open Source license) to make the Java source files available: this is best done in a top-leveljava
directory in the package – the source files should not be installed.
The ensure_*
family of functions can be used to enforce specific data types that are passed to a Spark routine. For example, Spark routines that require an integer will not accept an R numeric element. Use these functions ensure certain parameters are scalar integers, or scalar doubles, and so on.
In order to match the correct data types while calling Scala code from R, or retrieving results from Scala back to R, consider the following types mapping:
| From R | Scala | To R |
|---|---|---|
| NULL | void | NULL |
| integer | Int | integer |
| character | String | character |
| logical | Boolean | logical |
| double | Double | double |
| numeric | Double | double |
| Float | double | |
| Decimal | double | |
| Long | double | |
| raw | Array[Byte] | raw |
| Date | Date | Date |
| POSIXlt | Time | |
| POSIXct | Time | POSIXct |
| list | Array[T] | list |
| environment | Map[String, T] | |
| jobj | Object | jobj |
Most Spark extensions won’t need to define their own compilation specification, and can instead rely on the default behavior of compile_package_jars
. For users who would like to take more control over where the scalac compilers should be looked up, use the spark_compilation_spec
fucnction. The Spark compilation specification is used when compiling Spark extension Java Archives, and defines which versions of Spark, as well as which versions of Scala, should be used for compilation.


The dataset provides information on the US annual natural gas consumption by state level between 1997 and 2019, and aggregate level between 1949 and 2019
Units: Million Cubic Feet
us_total
A data.frame with 3 variables.
An integer, the year of the observation
A character, the US state indicator
A numeric, the yearly total natural gas consumption in a million cubic feet by state or US aggregate
US Energy Information Administration (EIA) website.
The dataset contains a summary of the total annual demand for natural gas in the US by state and total aggregate level. The data is available for the state level between 1997 and 2019, and for the US level between 1949 and 2019.
data("us_total") # Subsetting the total consumption in the US us_agg <- us_total[which(us_total$state == "U.S."),] us_agg <- us_agg[order(us_agg$year), ] head(us_agg)#> year state y #> 1034 1949 U.S. 4971152 #> 1035 1950 U.S. 5766542 #> 1036 1951 U.S. 6810162 #> 1037 1952 U.S. 7294320 #> 1038 1953 U.S. 7639270 #> 1039 1954 U.S. 8048504at_x <- seq(from = 1950, to = 2020, by = 10) at_y <- pretty(us_agg$y)[c(3, 5, 7)] plot(us_agg$year, us_agg$y, col = "#1f77b4", type = "l", frame.plot = FALSE, axes = FALSE, panel.first = abline(h = at_y, col = "grey80"), main = "US Total Natural Gas Consumption", xlab = "Source: https://www.eia.gov/", ylab = "Million Cubic Feet")mtext(side =1, text = format(at_x, digits=0, nsmall=0), at = at_x, col = "grey20", line = 1, cex = 0.8)mtext(side =2, text = format(at_y, scientific = FALSE), at = at_y, col = "grey20", line = 1, cex = 0.8)### Plotting the annual consumption in New England states # Subsetting the New England states ne <- c("Connecticut", "Maine", "Massachusetts", "New Hampshire", "Rhode Island", "Vermont") ne_gas <- us_total[which(us_total$state %in% ne),] # Reshape to wide ne_wide <- reshape(ne_gas, v.names = "y", idvar = "year", timevar = "state", direction = "wide") names(ne_wide) <- c("year",ne) # Reorder the data ne_wide <- ne_wide[order(ne_wide$year), ] # Set the plot y and x axis ticks at_x <- seq(from = 2000, to = 2020, by = 5) at_y <- pretty(ne_gas$y)[c(2, 4, 6)] # plot the first series plot(ne_wide$year, ne_wide$Connecticut, type = "l", col = "#073b4c", frame.plot = FALSE, axes = FALSE, panel.first = abline(h = c(at_y), col = "grey80"), main = "New England Annual Natural Gas Consumption by State", cex.main = 1, font.main = 1, col.main = "black", xlab = "Source: https://www.eia.gov/", font.axis = 1, cex.lab= 0.7, ylab = "Million Cubic Feet", ylim = c(min(ne_gas$y, na.rm = TRUE), max(ne_gas$y, na.rm = TRUE)), xlim = c(min(ne_gas$year), max(ne_gas$year) + 3))# Add the y and x axis ticks mtext(side =1, text = format(at_x, digits=0, nsmall=0), at = at_x, col = "grey20", line = 1, cex = 0.8)mtext(side =2, text = format(at_y, scientific = FALSE), at = at_y, col = "grey20", line = 1, cex = 0.8)text(max(ne_wide$year) + 2, tail(ne_wide$Massachusetts,1), "Massachusetts", col = "#118ab2", cex = 0.7)text(max(ne_wide$year) + 2, tail(ne_wide$`New Hampshire`,1) * 1.1, "New Hampshire", col = "#06d6a0", cex = 0.7)